About REST APIs

This is a general background information on the philosophy and design of REST APIs, and why they are used in the Journyx (JX) application programming interface (API).

For a quick summary of working with the Journyx REST API, see the Core Concepts page, or read the detailed tutorial.

Lessons from SOAP APIs

The Journyx application has long included the so-called "jxAPI" feature that provided a programming interface based on the now-archaic SOAP (Simple Object Access) protocol, and later WSDL and related "Web Services" standards.

About the legacy SOAP API

The Journyx SOAP API is still available for use and is documented here, but it is considered legacy technology and is not recommended for new development. Eventually, the SOAP API will be deprecated and phased out of the product.

Aside from being outdated, these standards largely ignored the semantics provided by the HTTP protocol which powers the web, and only used HTTP as a transport mechanism for a separate XML-based protocol. The nature of that protocol allowed developers to create their own wildly inconsistent semantics around the API signature (method/function and data structure specifications). In this case, semantics refers to the underlying ideas, patterns and use cases that informed the API design. For instance, how to create, read, update, and delete objects.

In other words, each area of the jxAPI is a unique creature with little consistency across different product areas in terms of semantics, naming conventions, data structures, and so on.

Furthermore, all new product features essentially had to be developed twice; once for the CGI-based web UI and again for the jxAPI and programmatic access. And then there were different jxAPI endpoints and sub-protocols that made things even messier, because not every endpoint supported every capability. Those jxAPI methods that did get created were often very low-level in nature, essentially exposing the raw DB data structures and expecting clients to know, understand and follow similar semantics as the CGI code, often with relatively little guidance from documentation. Therefore it would be difficult for a client developer or customer to understand how to use the jxAPI without a lot of trial and error.

Representational state transfer (REST) architecture

With the REST API we hope to change that, and learn from the experiences of the SOAP / WSDL world. A comprehensive definition of "REST" is out of scope for this document, but to summarize, it is a philosophy or methodology of how to interact with web-based services that leans heavily on the semantics provided by the HTTP protocol itself. It is not in itself a product or even a formal software specification or standard. It is what Wikipedia calls a "software architectural style." In effect, REST is the philosophy that web services should be designed following the same principles that make the web itself work so well.

One of the key benefits is that standardized semantics can be used throughout the system, and a client developer does not have to grapple with wildly different interfaces and semantics when working with different areas of the system. In general, the idea is to treat (publicly-facing) objects as "web resources" each with a unique address (the URL or uniform resource locator), and then use the methods provided by HTTP to create, read, update and delete these URLs. This set of actions is also known as the CRUD operations. These CRUD operations correspond to POST, GET, PUT, and DELETE in the HTTP protocol. The data objects themselves can represented in any format supported by the web, and can have multiple representations in different formats (e.g. XML, JSON, plain text, etc), each equally valid. However, at least in the Journyx API, objects are almost always represented in JSON (JavaScript object notation) format. Among other reasons, that makes the data easy to use inside JavaScript and indeed most other programming languages.

Basic REST principles

REST stands for representational state transfer, which refers to the core underlying concept: A "stateless" client/server system where various software resources (objects) can have a representation (JSON, XML, or anything else) that is transferred between the client and server with a uniform interface regardless of object type. The representation is not necessarily the thing itself, but an encoded description of it that the client can manipulate and send back to the server to change or update the state of the application.

Having a uniform interface based on the HTTP protocol helps to standardize how you interact with any object. The API representation may have little to nothing in common with how the resource is stored or represented internally in the software.

Web server APIs can follow any number of protocols or architectures, but an API is said to be "RESTful" when it incorporates REST principles into the core design. This means, among other things, that each HTTP request is a complete transaction, removing most of the notion of a separate server-side session state for each client. That transaction could deal with a longer-running process, such as initiating or checking for the completion of some larger task. It also implies a layered stateless system that works well with other components of the web, such as load balancers, caching proxies, and microservices.

Basic principles include:

• Resources are the primary entities and can be abstract in nature - a user, a project, a time entry, etc. Typically but not always, resources represent records (or sets of records) in a database or similar entities, but they can be almost anything.

• Resources can have representations in one or more formats such as XML, JSON, YAML, etc. The representation isn't the resource itself, but a view of it.

• Resources can be addressed in a unique way, through URLs / URIs.

• Clients can request changes in the server's state by creating or modifying a representation and sending it back to the server with an appropriate method (usually POST to create things, and PUT to modify them).

• As mentioned above, the representations that the API provides may have nothing to do with the actual internal storage format, field names, or anything else. In other words, the API representations can be made more "consumer-friendly" and do not need to directly reflect internal details and naming conventions, although that is often the easy road to take.

• Hypermedia as the engine of application state. This principle can help inform the design of representations so that each one contains links to all other resources and actions that are related to it. For example, you could start with a single root or base resource (URL) and then crawl the links to dynamically discover all actions available throughout the system without needing to hardcode this information in the client. There is some debate whether it's necessary to strictly follow this principle to be "REST-ful".

Contrasting SOAP and REST

Let's contrast the differences between the legacy SOAP / WSDL style and the REST philosophy.

Under the legacy jxAPI SOAP / WSDL architecture:

• The HTTP web protocol is just a transport method for XML documents that contain the actual API requests and responses. In fact, HTTP can be replaced entirely with other application-layer protocols.
• The HTTP methods and response codes were generally ignored or of minimal importance. For example, even server errors could return 200 OK responses, and the only indication of an error might be found by looking inside the XML document delivered in the response.
• Usually, only the two HTTP methods GET and POST were used, sometimes interchangeably.
• The URL of the request would generally be a protocol endpoint only, and would be the same in every request, no matter what those requests actually did.
• In general, the way you interact with the server is highly dependent on arbitrary details decided by whoever created the API method you want to use, and it becomes heavily reliant on good documentation. For example, the way to create or delete objects, or the techniques to query or filter lists of objects, could vary widely. These could be entirely unique to each section of the API.
• Client programs trying to use the jxAPI would often need relatively heavyweight libraries to handle both making HTTP requests as well as the XML parsing and related support code needed to facilitate requests. XML is a very complicated markup language in itself with all sorts of edge cases.
• Data structures and data types are dictated by the capabilities of both XML and the specific endpoint being used.

Under the REST API:

• The full HTTP protocol forms the core of the entire REST architecture.

• Our REST API likewise forms the core of the Journyx server architecture, in that the end goal is that the REST API is in effect the one and only way (or at least the main way) to get data into or out of the system, both for our own user interface as well as any other API clients or integrations.

  • In this way, most features only need to be developed once to support both web or mobile interfaces as well as programmatic API access.
  • There is only one set of business logic rules; those applied by the REST API. Therefore you automatically ensure that the web UI and programmatic APIs follow the exact same set of server-side business rules. (Client-side behaviors obviously can vary, such as different visual presentations in the mobile app vs web app.)

• The methods provided by HTTP (sometimes called verbs) translate directly to the server's data API - for example, GET will get an item, DELETE can actually delete a resource (object) on the server, POST is used to create a resource, PUT to modify an existing resource, and so on. There are a few other methods like PATCH that are sometimes used. A resource could be a user record, a project, or anything else, including less tangible things like a linkage between two other objects or some other piece of metadata.

  • You might think that being limited to the predefined methods of HTTP might severely limit the range of expression and the possible actions that you can take in the API. In realty, it is no limitation at all. The vast majority of interactions will fit neatly into this model, thus it becomes simpler to understand and work with for both the server API developer as well as the client developer.
  • Those use cases that don't neatly fit into the HTTP methods can usually be handled in a reasonable way as an edge case, or as a unique resource type (URL) that combines several other pre-existing types, or with special flags or request parameters.

• The URL of the request directly determines the type of object being addressed (e.g., a collection of users, a specific project, etc), and the method (verb) determines what happens to the object or collection being addressed.

• The HTTP response codes are directly relevant to using the API and the full expressive range of HTTP response codes can be used to signal different condition to the client (as well as other intermediaries such as load balancers or proxies). For example, a POST creation request may get a 201 Created response, instead of the more generic 200 OK. A 403 Not Authorized response is used to indicate lack of permission to do something.

• REST doesn't require the use of JSON at all, but it has become the de-facto standard object representation because it is convenient to work with in most languages. The primary data structure is a collection (an array or list) of objects. A resource object is simply a dictionary in Python terminology (a key to value mapping). This data structure is also called an "object" in JavaScript. In other languages it may be called a hashmap or associative array.

  • The values associated with each key can either be primitive values like strings and numbers, or else they can be other nested objects, or lists (arrays) of these types. This structure is largely dictated by the JSON format, but is flexible enough to accommodate a great many different use cases.

• In theory, any data representation format including XML can be used in a REST API, but in practice, we exclusively use the JSON format in our current API. Objects are represented as (potentially nested) dictionaries, and collections are arrays (lists) of objects.

• The REST philosophy goes much deeper that we have space to describe here. One example is how you can consider the entire API to be a collection of resources that you can discover in a web-crawling fashion by starting from a single root API object. Each object contains links to related objects, allowing you to explore and discover the full API in a more natural and interactive manner, rather than relying only on documentation. It at least allows for (doesn't mandate) a type of discoverability.

To summarize, adopting a REST philosophy means understanding and utilizing the full range of possibilities provided by the HTTP protocol and the architecture of the web in general.

Further reading

Here are a few more useful links with background information about REST principles and history.

Quick reads:

• RedHat's summary of REST architecture

• Codecademy - What is REST?

Deep dives:

• Azure - Best Practices for REST API Design. This is a great resources with lots of links to other useful information.

Next steps

You might want to start with testing out a few basic API calls using PyCharm's built-in tools by proceeding to the tutorial chapter.

The next section covering Core Concepts provides a high-level overview of the Journyx REST API and its key concepts and features.