Dumpleton Software Consulting Pty Limited OSE Version 7.0pl5 Python Manual 19 January 2003 Copyright 2001-2003 Dumpleton Software Consulting Pty Limited http://www.dscpl.com.
Table of Contents Table of Contents........................... 3 Manual Overview .......................... 7 Python Modules ............................. 9 Module Descriptions ....................... 10 Installation and Setup ...................... 10 Additional Information.................... 11 Logging Facility........................... 13 Logging a Message ......................... 13 Specifying a Log File ...................... 14 Specifying a Log Channel ...............
Table of Contents Service Lookup ...............................44 Handling Structured Types..............85 Service Reports.............................47 Servlet Framework .......................87 Publishing Reports ..........................48 Monitoring Reports .........................48 Lifetime of Reports .........................50 Identity of Subscribers ....................52 Existence of Publishers ...................54 Framework Overview......................87 The HTTP Daemon ...........
Managing User Sessions ............... 116 The XML-RPC Gateway............... 118 The SOAP Gateway ...................... 121 Using Multiple Gateways..............
Table of Contents 6
Manual Overview This manual covers the Python wrappers around the OSE C++ class library. The wrappers make available functionality related to the logging system, the real time events system, the service agent framework for creating distributed applications, the HTTP servlet framework and the RPC over HTTP interfaces. Python Modules Lists the available Python modules and their purpose. Includes brief details regarding installation and setup of the users environment.
Manual Overview 8 Service Agents Describes how to create service agents, add them to groups, subscribe to announcements regarding specific services or membership of specific service groups. Service Reports Describes how to subscribe to reports published by specific services. Ie., describes the publish/ subscribe functionality provided by the service agent framework. Service Requests Describes how to send requests to remote or local service agents and how to handle any response or error which results.
Python Modules OSE includes a number of Python modules. The main module is a wrapper around functionality provided in the OSE C++ class library. Those parts of the OSE C++ class library for which a Python wrapper are provided are the logging system, the real time events system, the service agent framework for creating distributed applications and the HTTP servlet framework.
Python Modules Module Descriptions The Python modules, their names and their purpose are described below. Module Purpose netsvc This is the main module and provides wrappers around the functionality of the OSE C++ class library. It includes all that is required for building distributed applications using the service agent framework. netrpc This module provides a client implementation of the RPC over HTTP protocol implemented by OSE called NET-RPC. netsvc.
Additional Information An OSE installation supports libraries for different architectures. In order that the shared libraries for your specific platform can be found by the Python module, you should ensure that the OSE_HOST variable is set to the same value it was set to when OSE was installed. For example: OSE_HOST=X86_LINUX If you want to be able to run the "spyon" debugger, your PATH environment variable should include the OSE bin directory. For OSE 7.
Python Modules 12
Logging Facility The logging facility provides you with a mechanism for generating and capturing messages generated by your application. These can be automatically saved to a log file, or intercepted and dealt with in some other way. The majority of functionality for this feature is provided by the OTC_Logger class in the OSE C++ class library. Some of the features of the logging facility are optional and controlled via environment variables.
Logging Facility Level Usage LOG_NOTICE Conditions that are not error conditions, but that may require special handling. LOG_INFO Informational messages. LOG_DEBUG Message that contain information normally of use only when debugging a program. To log a message, a handle to an instance of the Logger class is acquired and the "notify()" member function is called. import netsvc logger = netsvc.Logger() logger.notify(netsvc.
Specifying a Log File The string used to specify the name of a log file may incorporate the following special tags. Tag Purpose %h Will encode the hostname of the machine into the name of the log file. %p Will encode the process ID into the name of the log file. %Y Will encode the current year as 4 digits into the name of the log file. %y Will encode the current year as 2 digits into the name of the log file.
Logging Facility Specifying a Log Channel When logging a message, a log channel may also be specified. If the name of a log channel starts with a character other than an alphanumeric character, the message will not be displayed on the standard error output or appear in the log file. If it is displayed or captured in the log file, the name of the log channel does not appear anywhere in the message.
Exceptions in a Callback try: function() except SystemExit: raise except: netsvc.logException() sys.exit() The details of the exception are logged with level "LOG_ERROR" and a specific log channel is not specified.
Logging Facility 18
Program Setup As Python is an interpreted language, configuration of an application can be carried out by editing the actual scripts. In some circumstances however, it is still easier or more practical to rely upon a configuration database or environment variables. When using OSE this is especially the case, as an application can be a mix of C++ and Python code and configuration data may need to be accessible from code written in both languages.
Program Setup import netsvc import os netsvc.loadConfig("database.cfg") netsvc.mergeConfig("PWD",os.getcwd()) print netsvc.lookupConfig("PWD") A single configuration item can be removed from the database using the "removeConfig()" function. The configuration database can be completely emptied using the function "removeAllConfig()". The contents of the configuration database can be saved to a file using the "saveConfig()" function. netsvc.removeConfig("PWD") netsvc.saveConfig("database.
Naming Hierarchies file = "database.cfg" errors = netsvc.loadConfig(file,None) if errors: errors = "Error reading %s\n%s" % (‘file‘,errors) netsvc.Logger().notify(netsvc.LOG_DEBUG,errors) Naming Hierarchies If a naming hierarchy is required, the components of the hierarchy within the name should be separated by using a period. compiler.preprocessor.debug-level : 0 compiler.parser.debug-level : 1 compiler.code-generator.debug-level : 0 compiler.assembler.
Program Setup To lookup the value of an environment variable the function "lookupEnviron()" is used. If a new environment variable needs to be set, or an existing value changed, the function "mergeEnviron()" is used. Any changes to the environment variables will be visible immediately, but there is no way to get a list of all environment variables which are set. When a lookup is made but no such environment variable exists, the value None is returned. netsvc.mergeEnviron("PWD",os.getcwd()) print netsvc.
Process Identity id1 = netsvc.uniqueId(netsvc.ID_SHORT_FORMAT,"$SID?") The short format identifier is suitable for use within the context of a single process. Duplicates would only be encounterd if the incremental count of the number of identifiers exceeded what can be stored within a 32 bit integer value. If this were to occur, the counter would wrap around to zero and conflicts might thus arise if the existing identifier were still active.
Program Setup 24
Event Framework The main support for concurrency in the OSE C++ class libraries comes in the form of a mechanism for building event driven systems. This is based around a central job queue and a dispatcher, which takes successive jobs from the queue and executes them. To support real time systems, there also exist a number of event sources which will schedule jobs to trigger an agent to be notified when an event of interest occurs.
Event Framework jobs will be reclassified as standard jobs and subsequently executed. When scheduling a job, if jobs of the same type already exist, the new job will be placed at the end of the list of jobs of the same type. To schedule a job the dispatcher member function "schedule()" must be called, supplying the callback function and the type of job. To set the dispatcher running, the member function "run()" is called.
Scheduling a Job All that is occuring here is that when the "cancel()" member function is called, a flag is set. When the job is executed it will note that the flag is set and will not execute the callback function. If the callback function is a member function of a class, it is important to ensure that any reference to the instance of the Job class is destroyed when no longer required.
Event Framework cute()" member function, these will override any which may have been supplied when that instance of the Job class was created. Real Time Events The Python interface provides the ability to register interest in a number of real time events. These are program shutdown, one off alarms or actions, recurring actions, timers, signals and data activity on sockets.
Destroying Agents using the same identifier will first be cancelled. If you want to cancel all jobs scheduled using the "scheduleAction()" member function you should call the "cancelAllActions()" member function. Destroying Agents Ensuring that any outstanding job is cancelled, or deregistering interest in any event source, is important if you are endeavouring to destroy an agent object.
Event Framework The member function for setting an alarm is "setAlarm()" and that for starting a timer is "startTimer()". The first argument is the callback function, the second argument is the absolute or relative time and the third argument is an optional identifier for that alarm or timer. Scheduling an alarm or timer with an identifier matching that of an alarm or timer which hasn’t yet expired will cause that unexpired alarm or timer to be cancelled.
Socket Events class Object(netsvc.Agent): def __init__(self): self.scheduleAction(self.daily,"0 0 * * *","daily") self.scheduleAction(self.weekly,"0 0 * * Sat","weekly") self.scheduleAction(self.monthly,"0 0 1 * *","monthly") self.scheduleAction(self.yearly,"0 0 1 Jan *","yearly") self.scheduleAction(self.
Event Framework Other possible values for the third argument are SOCKET_POLLOUT and SOCKET_POLLPRI. The value SOCKET_POLLPRI is similar to SOCKET_POLLIN except that it relates to there being priority out of band data being available for reading. Out of band data is not a feature which is used much these days and isn’t implemented the same on all systems. It is probably best to avoid using out of band data.
Program Shutdown ating system is being shutdown. Other uses for program signals are to force an application to reread a configuration file. These three cases are typically indicated by the program signals SIGINT, SIGTERM and SIGHUP. A robust application should at least catch the first two of these signals and cause the program to shutdown gracefully. This may entail ensuring that any data is written out to files, removal of file locks, closing off of database connections etc.
Event Framework should call this member function. This member function can also be called when an external signal intended to shutdown the program is received. Doing this in the latter case means you don’t need to have separate code for the two different cases. If an agent is interested in the fact that the program is being shutdown, it can call the "subscribeShutdown()" member function, supplying a callback function to be called when such an event does occur.
Program Shutdown itly provide the time delay as an argument. If this is done, the argument should express the number of full or partial seconds as a floating point value. Using a time delay is a useful starting point, as it provides a means of defining an upper bound on the amount of time you wish to allow the system to run before it is stopped.
Event Framework Note that whatever mechanism is used to initiate program shutdown using these features, messages will be displayed via the logger indicating that shutdown has been scheduled and that it has arrived. Additional messages will be displayed via the logger when the shutdown process is suspended and resumed.
Service Agents The service agent framework in OSE provides request/reply and publish/subscribe features similar to that found in message oriented middleware packages. Unlike most of the available packages, the service agent framework does not have a flat namespace with respect to naming, but uses an object oriented model, with each service having its own namespace with respect to subject names for subscriptions and request method names.
Service Agents The major classes in the OSE C++ class library involved in providing this functionality are the OTC_SVBroker, OTC_SVRegistry and OTC_EVAgent classes along with various event classes. In a distributed application the OTC_Exchange class comes into play along with the various classes used to implement the interprocess communications mechanism. Service Naming When using the C++ class library, implementation of a service agent entails the use of a number of different classes together.
Service Audience Service Audience When you create a service, the existance of that service will be broadcast to all connected processes. If you wish to restrict visibility of a service to just the process the service is contained in, or a subset of the connected processes, a service audience can be defined. To define the service audience, an extra argument needs to be supplied to the Service base class when it is initialised.
Service Agents Service Groups When a service agent is created, the name of the service is notionally listed in a global group. In respect of this global group, unless you track the coming into existance of every single service agent, there is no way to make conclusions about a subset of services.
Service Registry Any service agent may make queries against its local service registry and get back an immediate result which reflects the current state of the service registry. A service agent may also subscribe to the service registry or aspects of it and be notified in real time of changes made to the service registry.
Service Agents Member functions of a service binding object which may prove useful include "serviceName()", "agentIdentity()", "serviceAudience()", "processAddress()" and "serviceLocation()". Of these, "serviceLocation()" returns either "SERVICE_LOCAL" or "SERVICE_REMOTE", giving an indication if the service is located in the same process or a remote process. The "processAddress()" member function will return an internal address relating to the actual process the service is located in.
Service Announcements Service Announcements If a service agent subscribes to the registry using a specific service name, the service agent will be notified when any service with that name becomes available or is subsequently withdrawn. When subscribing to the registry using a specific service name, no notification is given regarding groups that those same services may join. class ServiceMonitor(netsvc.Monitor): def __init__(self,name): netsvc.Monitor.__init__(self) self.subscribeServiceName(self.
Service Agents Group Announcements If a service agent subscribes to the service registry using a specific service group, it will be notified when any service joins or leaves that group. Notice that a service has left a particular group will also be be notified when the service is withdrawn and the service hadn’t explicitly left the group before hand. The member functions relating to service group subscriptions are "subscribeServiceGroup()" and "unsubscribeServiceGroup()".
Service Lookup class PollingService(netsvc.Service): def __init__(self,name,period=60): netsvc.Service.__init__(self) self._name = name self._period = period self.initiateRequests("poll") def initiateRequests(self,tag): bindings = self.lookupServiceName(self._name) for binding in bindings: service = self.serviceEndPoint(binding) # presume remote service provides uptime method id = service.uptime() self.processResponse(self.handleResult,id) self.startTimer(self.initiateRequests,self.
Service Agents 46
Service Reports When using the service agent framework, in addition to being able to subscribe to the service registry in order to receive announcements regarding the existence of services, it is also possible to subscribe to actual services. When subscribed to a service, if that service publishes a report with a subject matching the subscription, the subscriber will automatically receive it.
Service Reports Publishing Reports If a service agent needs to publish a report, the member function "publishReport()" is used. In publishing a report, it will generally be the case that a service agent does it without caring who may actually be subscribed to that report. This is often referred to as anonymous publishing and results in a more loosely coupled system which can adjust dynamically to changes.
Monitoring Reports a subscription of "system.*" will match "system.time" and "system.statistics.users", but not "system". To subscribe to any reports from a particular publisher, "*" would be used. Note that the subscription pattern described here is the default. It is actually possible within the C++ implementation of a service agent to override the default and supply an alternate matching algorithm.
Service Reports class Subscriber(netsvc.Service): def __init__(self): netsvc.Service.__init__(self) # subscribe to the service group "publishers" self.subscribeServiceGroup(self.announce,"publishers") def announce(self,binding,group,status): if status == netsvc.SERVICE_AVAILABLE: # now subscribe to service agent which is member of group self.monitorReports(self.report,binding,"system.*") else: self.ignoreReports(binding) def report(self,service,subject,content): binding = self.currentReport().
Lifetime of Reports For such cases, it is possible to supply an optional lifetime for a report. That is, a time in seconds for which the report should be cached by the publishing service. When such a value is supplied, if a subscription arrives within that time, it will be sent a copy of that report. If a value of "-1" is supplied for the lifetime, it will effectively cache the report indefinitely. # publish and cache indefinitely self.publishReport("system.
Service Reports Although "purgeReports()" exists specifically to deal with potential performance issues in a very limited number of cases, the "expireReports()" and "expireAllReports()" member functions are useful where a service may have reset itself and it was necessary to discard all cached reports so that new subscribers didn’t receive them. Identity of Subscribers In most circumstances the identity of a subscriber is not important, however, such information can be quite useful in a few circumstances.
Identity of Subscribers A further use of the mechanism for identifying a subscribers identity, is so that subscriptions can be tracked and for processing or interception of data only to be undertaken while there are subscribers interested in the results. This avoids unnecessarily publishing reports when it is known there would be no one to send them to. class LogMonitor(netsvc.Service): def __init__(self): name = "logmon@%s" % netsvc.processIdentity() netsvc.Service.__init__(self,name) self.
Service Reports subject corresponding to a log channel, the log messages on that log channel will be intercepted and published. This can be useful as a remote debugging mechanism and will not unnecessarily load the process as information is only being captured and published when it is actually required. Existence of Publishers When a subscription to a service is made, if the service holds any cached reports with a subject matching the subscription, the subscriber will receive them immediately.
Existence of Publishers essary to track the lifetime of each. It is also useful where it might be necessary to immediately send off a request to each service agent to obtain information not available via published reports.
Service Reports 56
Service Requests The ability within the service agent framework to find out what services exist and the ability of a service agent to publish reports can be useful in itself, but more often than not one wants to make a specific request of a service to perform some action. In most cases such an action would result in a response, be it the return of data related to the request being made or an error indication.
Service Requests When invoking "serviceEndPoint()", the member function needs to be supplied with either a service binding object for the particular service agent to which you wish to send the request, or the name of the service. When a service name is supplied, a lookup will be made against the service registry and the first service agent found with that service name will be used. To invoke the request against the remote service agent, the service endpoint object is used as a proxy.
Handling a Response for binding in bindings: service = self.serviceEndPoint(binding) if service: service.reset() Handling a Response When you send a service request, you do not get an immediate response back. That is, the call is asynchronous. If you want to be able to capture any response generated from a request, you need to capture the conversation id associated with the request and then register a callback to handle the response.
Service Requests Identifying a Response If a callback is being registered to handle the response from multiple service requests, you will most likely need to be able to identify to which request a response belongs to. To get the conversation ID of the original request, the "conversationId()" member function can be called. class Client(netsvc.Service): def __init__(self,name): netsvc.Service.__init__(self,"","") bindings = self.lookupServiceName(name) for binding in bindings: service = self.
Detecting a Failure id = service.reset() self.processResponse(self.resetResponse,id) print "request",binding.agentIdentity(),id def resetResponse(self): print "result" In addition to "conversationId()" the member function "currentResponse()" is also available. This member function returns an object providing both the "conversationId()" and "sender()" member functions.
Service Requests if failure.origin() == "netsvc" and \ failure.error() == netsvc.SERVER_METHOD_UNAVAILABLE: # method didn’t exist As an alternative to using the "conversationId()" member function to obtain the conversation id of the failed request, if the callback accepts a single argument, the conversation id will be passed as an argument to the callback function. class Client(netsvc.Service): def __init__(self,name): netsvc.Service.__init__(self,"","") service = self.
Lack of Response Lack of Response When you send a request, there is no gaurantee that the remote service agent hasn’t been destroyed even before it receives your request. If a remote service agent delays sending an immediate response to your request, the problem might also arise that the remote service agent is destroyed before it completes the response. Finally, an intermediate process relaying your request might be shutdown or crash meaning either the request or response is lost.
Service Requests self.processFailure(self.uptimeFailure,id,60) def uptimeResponse(self,result): print result def uptimeFailure(self,id,error,description,origin,details): if origin == "netsvc" and error == netsvc.CLIENT_REQUEST_TIMEOUT: # timeout occurred When a timeout occurs, it will be notified as a request failure. The timeout should be the maximum number of seconds to wait. The callback will be automatically deregistered and if the response did subsequently arrive it would be ignored.
Generating a Failure The reason for requiring that methods be explicitly exported is that it would usually be quite dangerous to allow open access to all member functions of a class. This is because any class is likely to implement methods to assist in intermediate stages of processing a request. Providing automatic access to such member functions could compromise the operation of the class.
Service Requests raised, you should avoid an except clause which catches all exceptions in any code which encloses code which might call "abortResponse()". Alternatively, you should explicitly pass on exceptions of type ServiceFailure. try: self.execute(...) except netsvc.ServiceFailure: raise except: self.abortResponse(...
Delaying a Response When the member function "suspendResponse()" is called, a callback function should be supplied as argument which finalises the request and returns the appropriate result. The callback passed to "suspendResponse()" will only be called when the "resumeResponse()" method is called at some later point in time. In particular, you would call "resumeResponse()" once you have collected together all the data which forms the result for the original request. class DatabaseProxy(netsvc.
Service Requests Note that "suspendResponse()" and "resumeResponse()" were only added in OSE 7.0pl5 and are a layer on top of the "delayResponse()" method which only performed the single operation of raising the exception of type DelayedResponse. The newer functions should make the task of implementing a delayed responese easier, so if you are using "delayResponse()" you should change your code to use the newer functions.
Invalid Request Method client = self.currentRequest().sender() self._count = self._count + 1 name "%s/%d" % (self.serviceName(),self._count) session = Session(name,client) return name Such a mechanism as described can’t be used if such a request to create a session may originate over an RPC over HTTP connection. This is because the service agent which acts as proxy for the request is transient and will be destroyed once the request has completed.
Service Requests that the result is returned immediately. Note that since the response is immediate, you can’t call a method which itself would try and use a delayed response. class DatabaseProxy(netsvc.Service): def __init__(self,name="database-proxy") netsvc.Service.__init__(self,name): self.exportMethod(self.tables) self._active = {} def tables(self): service = netsvc.LocalService("database") return service.
Message Exchange The features of the service agent framework may be used standalone within a single process or across a set of connected processes. That is, use of the service agent framework is not dependent on a process being able to connect to a central message exchange process. When combined with the HTTP servlet framework and RPC over HTTP interface, a single process may be more than adequate for many applications, especially simple web based services.
Message Exchange Exchange Initialisation To create a message exchange endpoint in a process, the Exchange class is used. When creating an instance of the Exchange class it is necessary to specify whether it is performing the role of a message exchange server or that of a client. A message exchange server is a process which takes on the role of being a hub for message distribution.
Service Availability er endpoint and a new client connects, a subscriber to that service in the client will be notified that the service is available. Similarly, any service within a client will become visible from the server as well as other connected clients. Although the service is located in a separate process, the same service registry interface is used to subscribe to the presence of the service.
Message Exchange print "%s originally started at %s" % (publisher,str(content)) elif subject == "system.time": print "%s was still alive at %s" % (publisher,str(content)) dispatcher = netsvc.Dispatcher() dispatcher.monitor(signal.SIGINT) exchange = netsvc.Exchange(netsvc.EXCHANGE_CLIENT) exchange.connect("localhost",11111,5) dispatcher.run() The only difference is that a message exchange client has been added to each, the actual services are identical to what they were when used in the same process.
Authorisation of Clients Overriding this method can be useful purely for logging purposes, but might also be used in a client process to trigger an announcement to activate the function of the process upon a connection becoming active. Consequently, the operation of a client process could be suspended or the process shutdown when no active connection could be established or the connection lost.
Message Exchange When an application is distributed across multiple machines, it is often the case that even if one machine were to be shutdown, the processes on a different machine might be able to quite happily keep operating so long as they could still communicate. To support this, a means of setting up a distributed version of the message exchange server is provided. In this arrangement, each machine has its own message exchange server, with each message exchange server connected to all others.
Multiple Exchange Groups With respect to service visibility, a message exchange endpoint will only pass information about services if the service audience is "*", or if the service audience is the same as the exchange group. The only exception to this is when the exchange group is empty. In that case, an empty service audience will still restrict visibility of a service to its own process.
Message Exchange tached to the default exchange group. Back end services will still be able to see any services on the default exchange group which had a service audience of "*". Note that different exchange groups should not overlap. That is, they should only ever share at most one process with any other exchange group. In effect, exchange groups when used should form a hierarchy.
Message Encoding As the service agent framework is designed as a distributed system covering multiple programming languages, it is necessary that any data being passed around within a report, request or response be serialised into a form suitable for transmission as part of a message. At present the encoded form of the data uses a subset of XML. That is, it would qualify as being XML, however to make the implementation easier, the code for decoding such messages will not accept arbitrary XML.
Message Encoding Supported Data Types Communication between services is mediated through a layer of code which is written in C++. The only exception to this is when the LocalService class is used as a proxy to send a request to a service in the same process which is also implemented in Python.
Mapping of Scalar Types Type Format Sample DateTime YYYY-MM-DDThh:mm:ss 2001-12-25T23:59:59 Time hh:mm:ss 23:59:59 Duration PnDTnHnMnS P1DT23H59M59S For the date and time types, the current Python implementation does not do any checking to determine if the supplied values are valid, but will pass them as is. Note that the XML Schema Datatypes specification does allow for a timezone in a date and time, but it is recommended that all date and time values be sent as UTC.
Message Encoding Python Type Encodes To XML Type netsvc.Time xsd:time netsvc.Duration xsd:duration If a service is implemented using the OSE C++ class library directly, different size versions of the integer and floating point types are available and can be generated in the serialised form of any data. A consequence of this is that when converting any data from its serialised form into instances of Python types, a broader range of possible values types need to be accommodated.
User Defined Types "xsd:hexBinary" will also be added at some point in the future as well. If you wish to send a Unicode string, you should convert it into a string using UTF-8 encoding. User Defined Types The intent with the XML Schema Datatypes specification is that additional scalar data types can be introduced by assigning a new name scoped within a distinct namespace. In respect of the types defined by this specification, the namespace "xsd" is used.
Message Encoding To add a new mapping at global scope the functions "encoder()" and "decoder()" should be used to register functions to do the appropriate conversions. When registering the encoder, the first argument should be either the type object or class object as appropriate. When registering the decoder, the first argument should be the qualified name you have given the type. The encoder function which you register should accept a single argument, that being an instance of your type.
Handling Structured Types ues of the correct type before hand. A service may also override the default decoders for extended types such as the date and time types if desired. Handling Structured Types The encoding mechanism for data does not provide a way of adding support for your own structured types, whereby the type of that object can also be transmitted. All objects need to be able to be converted into instances of scalar types, dictionaries, tuples or lists.
Message Encoding 86
Servlet Framework The HTTP servlet framework can be used to provide a window into your application. A number of predefined servlets are provided or you can create your own. You can also create your own server objects to map the servlets to appropriate parts of the URL namespace. Alternatively, a number of predefined server objects can be used for common tasks such as serving up files from the filesystem, or provision of RPC over HTTP services.
Servlet Framework to provide a servlet to handle the actual request. If no server object is found corresponding to that portion of the URL namespace, or the server object is not able to provide a servlet to handle the request, a HTTP error response is returned to the client indicating that the resource corresponding to the supplied URL could not be found. Where an appropriate servlet to handle the request is found, the session manager will initially pass off to the servlet the details of the request.
The File Server daemon.start() dispatcher.run() When a HTTP server object is registered, the first argument to the "attach()" member function should be the path under which resources made available by the HTTP server object are accessible. Except for the root directory, the path should not include a trailing "/". The path should also be in normalised form. That is, it should not include consecutive instances of "/" within the path, or include the path components ".." or ".".
Servlet Framework When an instance of the FileServer class is created, it must be supplied with the filesystem directory from which files are to be served. The server object utilises the Python mimetypes module for determining file types. The file type associated with an extension can be overridden, or knowledge of additional file types can be added using the "map()" member function. filesrvr = netsvc.FileServer("/home/httpd") filesrvr.map(".
User Authorisation If access to a particular client is disallowed, the connection will be dropped immediately. The client will not receive any form of specific HTTP error response indicating why the connection has been closed. Note that this mechanism blocks a client from accessing any part of the URL namespace for that HTTP daemon.
Servlet Framework return netsvc.ErrorServlet(404) daemon = netsvc.HttpDaemon(8000) server = HttpServer() daemon.attach("/test",server) daemon.start() The job of the "servlet()" member function is to create an instance of a HTTP servlet capable of handling a request made against a specific URL. When the "servlet()" member function is called it is supplied with the HTTP session object. The session object provides access to details of the request, including the server root and servlet path.
The Redirect Servlet The Redirect Servlet The redirect servlet as implemented by the RedirectServlet class would be used when it is necessary to redirect a HTTP client to an alternate resource. In addition to the HTTP session object, it should be supplied the URI of the resource to which the HTTP client is to be directed. By default, the HTTP response code will be "302", indicating the resource has been temporarily moved. This can be explicitly indicated by using the value "REDIRECT_TEMPORARY".
Servlet Framework becomes congested. Although the servlet may be forced to wait before it can send more data, any other jobs in the event system will still be serviced, including other HTTP requests. Logging of Requests By default no information is logged about requests. If you wish to log what requests are being made against your application using the HTTP servlet framework, you need to set the environment variable "OTCLIB_HTTPLOGCHANNEL" to the name of the log channel to record the information on.
Servlet Objects To make the most of the HTTP servlet framework it will be necessary to create your own servlets for interacting with your application. Servlets can be written to handle basic requests against a resource, or requests where form data is supplied. Special purpose servlets which process arbitrary content associated with a request may also be created.
Servlet Objects if self.requestMethod() != "GET": self.sendError(400) else: self.sendResponse(200) self.sendHeader("Content-Type","text/plain") self.endHeaders() self.sendContent("Hi there.") self.endContent() The major member functions of the HTTP servlet class used to interrogate the details of the HTTP request are "requestMethod()", "requestPath()" and "queryString()". It is these methods you would use to determine what the HTTP servlet is to do.
Delaying a Response connection. Negotiation of persistent connections between the HTTP client and server is managed by using special HTTP request and response headers. Where possible the session manager will undertake to maintain persistent connections without you needing to take any special actions. This is done as a result of the session manager inserting on your behalf the special headers as appropriate when you call the "endHeaders()" member function.
Servlet Objects Destruction of Servlets The destruction of a servlet can come about as a result of two situations. The first situation is where a servlet handles a requests and generates a response, whether that be successful or otherwise. The second situation is where the HTTP client closes the connection before the servlet has sent a complete response. The fact that the actions of a servlet may need to be aborted before it has finished complicate the destruction of a servlet.
Processing Content been derived from the Agent class, this would include calling the "destroyReferences()" member function. Processing Content If the function of a HTTP servlet entails that the content associated with a request be processed in some way, it will be necessary to override the "processContent()" member function.
Servlet Objects def processContent(self,content): self._content.append(content) self._contentLength = self._contentLength + len(content) if self._contentLength >= self.contentLength(): self._environ["REQUEST_METHOD"] = self.requestMethod() self._content = string.join(self._content,"") self._content = self._content[:self.contentLength()] fp = StringIO.StringIO(self._content) try: form = cgi.FieldStorage(headers=self._headers, \ environ=self._environ,keep_blank_values=1,fp=fp) self.
Slow HTTP Clients password = self.field("password") if self.authenticateUser(user,password): self.sendResponse(netsvc.REDIRECT_TEMPORARY) self.sendHeader("Location",self.serverRoot()) self.endHeaders() self.endContent() else: self.sendError(400) else: self.sendError(400) def authenticateUser(self,user,password): # ... The existance of a field can be determined by calling the "containsField()" member function. The member function "field()" can then be called to retrieve the value for the field.
Servlet Objects self._batch = None self._total = None self._count = 0 self._job = netsvc.Job(self.generateContent) def destroyServlet(self): FormServlet.destroyServlet(self) self._job.cancel() self._job = None def handleRequest(self): if not self.containsField("batch") or \ not self.containsField("total"): self.sendError(400) else: try: self._batch = int(self.field("batch")) self._total = int(self.field("total")) except: self.sendError(400) else: self.sendResponse(200) self.
Slow HTTP Clients Note that the Python wrapper around the C++ implementation of the HTTP servlet class performs buffering of content and will only pass content onto the C++ implementation when a set amount has been exceeded or the end of content has been indicated. If you suspend sending of further data, so that a HTTP client will see content produced so far, you may wish to flush out any buffered data by calling the "flushContent()" member function.
Servlet Objects 104
Servlet Plugins When using a file server object with the HTTP servlet framework, it is possible to associate a special purpose handler or plugin with requests against files with a particular extension. When a request is made against such a file, the plugin is used as an intermediary for the creation of a servlet to handle that request. The plugin can return a servlet which was loaded into the application at startup, or might also load the servlet from the file or otherwise generate a servlet on the fly.
Servlet Plugins The effect of this registration will be that whenever a file with extension ".py" is requested by a HTTP client, the plugin object will be executed as a callable object, with the HTTP session object and the name of the file being passed as arguments. In this case, the PythonPlugin object will import the file as if it is a Python module, obtain from it a reference to the HTTP servlet and then create an instance of the HTTP servlet to service the request.
Module Caching __servlet__ = ServletProxy() Note that an instance of the servlet is created for each request. That is, unlike other similar systems available for Python, an instance of a servlet object is not cached and reused. If you need to maintain state between requests, such information should be factored out into a distinct service agent object.
Servlet Plugins Note that a module imported in this way can use the same mechanism to import further modules with the dependence on those additional modules also being considered when the initial file is requested. Be aware however, that if files are located on a different machine to that which the application is running on and the clocks are not sychronised properly, updates may not always be detected correctly.
Plugin Aliasing a servlet. In this later case, the name of the resource can remain the same, and no references to the resource need to be changed. To facilitate use of an alias, an optional argument can be supplied to the "plugin()" member function defining the alternate extension the resource should be identified with. If you wanted all servlet files accessible using a particular instance of a file server object to be accessed using a ".html" extension instead of the ".py" extension, the string ".
Servlet Plugins 110
Remote Access The service agent and message exchange framework operate based on the concept of processes which are a part of a distributed application being permanently connected together. This model works fine on corporate networks, but is not always practical when run across the Internet. One drawback of this approach is that it is often necessary to open up special ports on a corporate firewall to permit access.
Remote Access Even if a new protocol comes along, it is a relatively simple matter to incorporate yet another gateway, again without you having to make modifications to the core of your system. The RPC Gateway The gateway which accepts an RPC request is actually an instance of a HTTP server object. The gateway will accept a request and based on the URL determine which service the request applies to.
The Client Application In this example, any HTTP request made using a URL whose path falls under the base URL of "http://localhost:8000/service/", will be regarded as being a NET-RPC request. The name of the service which a request applies to is determined by removing the base URL component from the full URL. The full URL used to access the service in this example would therefore be "http://localhost:8000/service/validator".
Remote Access return host in self._allow: class RpcGateway(netsvc.RpcGateway): def __init__(self,group,users=None): netsvc.RpcGateway.__init__(self,group): self._allow = users def authorise(self,login,password): return self._allow == None or \ (self._allow.has_key(login) and \ self._allow[login] == password) users = { "admin": "secret" } port = 8000 group = "web-services" httpd = HttpDaemon(port) rpcgw = RpcGateway(group,users) httpd.attach("/service",rpcgw) httpd.
User Defined Types User Defined Types The NET-RPC protocol supports all the types supported by the service agent framework, as well as the concept of user defined scalar types. That is, if a service responds with data incorporating additional scalar types, they will by default be passed back as instances of the Opaque type, where the "type" attribute gives the name of the type and the "data" attribute the encoded value.
Remote Access Managing User Sessions A common practice with web based services is to have a request initiate a unique session for a user. Having opened the session, any requests will then be identified with that session, with information regarding the session potentially being cached on the server side until the session is closed.
Managing User Sessions plicitly close off the session, it would be automatically closed after a period of 60 seconds of inactivity, or whatever period was defined when the session was initiated. An implementation of the database cursor service for this example might be as follows. class Cursor(netsvc.Service): def __init__(self,name,cursor,timeout): netsvc.Service.__init__(self,name) self.joinGroup("database-services") self._cursor = cursor self._timeout = timeout self._restart() self.exportMethod(self.
Remote Access self.cancelTimer("idle") self.destroyReferences() return 0 Using the "netrpc" module to access the service, a client might be coded as follows. In this case a separate cursor is created in relation to the queries made about each table in the database. import netrpc url = "http://localhost:8000/database" service = netrpc.RemoteService(url) tables = service.execute("show tables") timout = 30 for entry in tables: table = entry[0] print "table: " + table name = service.
The XML-RPC Gateway port = 8000 group = "web-services" httpd = netsvc.HttpDaemon(port) rpcgw = netsvc.xmlrpc.RpcGateway(group) httpd.attach("/service",rpcgw) httpd.start() dispatcher.run() If you do decide to rely upon the XML-RPC protocol instead of the NET-RPC protocol, you will be constrained as to what types you can use. This is because the XML-RPC protocol has a more limited set of core types and is not type extendable as is the NET-RPC protocol.
Remote Access # Explicitly specify use of Python implementation. rpcgw3 = netsvc.xmlrpc.RpcGateway(group,variant="python") Both these Python and C++ implementations of the routines for handling the XML-RPC protocol are supplied with OSE.
The SOAP Gateway The SOAP Gateway Yet another alternative to XML-RPC is the SOAP protocol. A starting point for SOAP is the site "http:/ /www.develop.com/soap". Although SOAP is newer than XML-RPC and is notionally type extendable, it actually has some more significant limitations than XML-RPC. As with XML-RPC, it is only recommended that you use this protocol in preference to the NET-RPC protocol if you really have to. In doing so you will need to write your code keeping in mind these limitations.
Remote Access port = 8000 group = "web-services" httpd = netsvc.HttpDaemon(port) rpcgw = netsvc.soap.RpcGateway(group) httpd.attach("/service",rpcgw) httpd.start() dispatcher.run() To complement the SOAP gateway, a SOAP client is provided in the "netrpc.soap" module. This client is only suitable for use against SOAP based web services which rely on positional arguments. Most web services use WSDL and therefore require named parameters, making the client unsuitable in those cases. import netrpc.
Using Multiple Gateways called "ServiceFailure". All elements will be qualified in the OSE namespace. Inadequate prior art has been found as to the most appropriate way to make use of the detail element, so it may be necessary to change this in the future if necessary. Using Multiple Gateways Because it is possible to attach multiple HTTP server objects to a particular instance of a HTTP daemon object, you aren’t restricted to having only one instance of an RPC gateway.
Remote Access of also pushing a lot of the security issues onto the main web server where they are more often than not easier to manage and deal with. If it was necessary to expose some part of the application direct to outside users, one approach might be to make use of the Apache "mod_proxy" module rather than directly exposing the application.
