Modularity is important to promote separation and code re-use, so that each individual module, such as HTTP parameter parser, could be loaded independently by other programs with a short name 'web/cgi/module'.
Each module should distinguish between private names and public interface, marked with scriptdoc NB.* comments.
Compliance with API design guidelines is essential, addressing the needs of three levels of programmers. For example, in the same area of HTTP parameters:
Opportunistic: it provides ready to use parameter accessors handling default values, type conversions and lists
Pragmatic: allow to substitute custom paramer provider or handler in place of standard, as in JWebServer/HttpParser
Systematic: provide access to raw HTTP parameters in their original sources for GET or POST, while possibly hiding the lowest protocol of whether it's stdin, environment variable or socket.
Programmatic interfaces for application developer.
- expose CGI environment variable as high-level standard set of accessors
- hiding platform specifics, e.g. SCRIPT_URL vs SCRIPT_NAME, etc.
provide physical paths with jpath aliases, e.g. ~CGI for root of current application, ~Page for location of the current page, etc.
- high-level accessors to HTTP parameters regardless of source (QUERY_STRING or stdin) with support for default values and type coersing in quasi-declarative form
handle multipart/form-data (for upload)
- provide key-value dictionary of parameters with given prefix, possibly coersed to numeric, etc.
- provide list of parameters with a given same name, as from a set of checkboxes, possibly coersed to numeric, etc.
- convert a set of parameters into a series of input fields or a URL string
- provide a accessor for request HTTP Headers
- provide a storage for output HTTP Headers
- automatically calculate output Content-Length
- maintain output Content-Type
- maintain output Content-Encoding
- provide stream-like interface for emitting string and typed data
- provide binary output, hide Windows text mode stdout issue
- support buffered mode with clearing and premature end of response
- support automatic encoding, specified in HTTP header
- support HTML Writer interface from utilities
handle errors: with top level try/catch
- error logging
- output diagnostics of unhandled errors according to content type
- provide normal termination and programmatic abnormal termination as
special types of throw
- provide a accessor for request cookies
- provide a storage for output cookies
- automatic encryption and decryption, possibly delegated to other addon
- automatically forward persistent cookies
Optional session "object" support
- maintain a session between requests
- support different session methods: cookies, hidden input field, scrambled container
- provide accessor to save and get different string values
Static functionality for web-related processing, such as parsing, encoding and formating. Some of the functionality could be delegated to other addons.
HTTP Parameter Processing
- decode and encode string with URL encoding
parse and generate URL query with a matrix with shape N,2
- split and combine URL with protocol, address, port, path, query and hash parts
- split request and combine response with headers, body and mime parts
- encode and decode text using HTML entity encoding
- convert between different character encodings: Unicode, UTF-8, standard national
- declare a list of standard mime type for validation, mapping of file extensions and support of other content generation modules, such as image converters
- decode and encode mime multipart messages
This is meant to be used by non-web server programs like GUI HTML formatters, etc. Possibly this should be a separate addon: web/html.
- programmatic generation of generic HTML elements
- automatic escaping and encoding of attributes
- verbs for some concrete HTML elements
- bulk generation: table from matrix, HTML list from boxed list, etc.
- HTML Writer interface with different output writers: string, file, generic output writer
parse and generate HTTP headers with a matrix of shape N,2
A set of handlers for executing high-level API frameworks such as JHP, Web Services.
Handlers are selected based on configuration and current file extension or file name, etc.
Generic handler for low-level handler implementation is provided for J scripts with .ijs extension. Developers can implement handlers for output images and other specific formats.
- Middleware Execution
invoked by CGI integration layer, see Integrated execution
Interfacing with low-level web-server protocols: standard (default) CGI, enhanced CGI (FastCGI, SCGI), J stand-alone Web Server, as well as abstract interface with integration adapters created in the future. E.g. ASP.NET request handler can act as such adapter in a way similar to FastCGI/SCGI invoking J via COM or DLL while leveraging IIS for load-balancing, multithreading, etc.
- configuration: reading and exposing
- error handling
- selecting adapter
- executing handler
- Integrated execution
- read CGI config file
- select CGI adapter
- identify script and select middleware handler
- setup CGI services for handler and application code
- setup error handling try/catch block
execute handler, see Middleware Execution
- process errors
- generic logging
- finalize execution
- CGI exit code depending on specific adapter
- integration interface should be very thin, delegating most of the HTTP processing to the CGI framework, and only implementing the low-level protocol, such as sockets, etc.
- integration is configurable
- much of the CGI implementation and all of the application code should be isolated from the choice of particular low-level adapter
- error reporting
- raw environment variable access: resolving and platform-specifics is done by higher level CGI
- input stream: buffering reading, parsing, splitting are done by higher-level interfaces
- output stream: buffering, combining, encoding, etc. done by higher interfaces
Setting up the front-end web server, J system, and CGI framework for mutual interaction.
Vendor-specific host web server procedures
- register specific web application or document root
- register a file extension for CGI framework, while leaving other files to be handled by host web server
- option of a dedicated instance of web server, such as "second" in JHP
local configuration files, e.g. .htaccess, web.config
- specifics of web user and permissions settings to allow certain functionality, such as OpenGL under Mac OS X
- boot code to recognize CGI mode
- delegating to CGI framework
Configuration file(s) with settings for various layers of CGI framework.
- Main config file should be located near J system to be found by the CGI boot code. Other, application-specific files, may be located in folders on the path from the web root to the current script folder.
- The following sections or properties can be configured