Building a Rails app without a database

At Katalyst, we often encounter intriguing challenges. A client approached us to implement a new business application (B2B) for their existing customer base. The client is a large enterprise that operates within a complex internal business environment. Their needs required balancing various requirements: cost-efficiency, robust security measures, and integration with internal services rather than building from scratch. 

We began with the project ideation phase, finding ways to reduce complexity and scope in our implementation. Given that data would be sourced from their existing legacy system, we proposed a different approach: an application that doesn't use any typical storage. No database or filesystem..

A benefit of building without storage is a reduction in ongoing costs for the client, as this approach eliminates the overhead of compliance and audit costs incurred by a financial services provider when maintaining a "system of record."

To successfully address our client's requirements, we aimed to avoid a frontend-heavy implementation, aligning with the Rails philosophy of server-side logic with JavaScript enhancements. Additionally, the application needed to interface with an existing web login service backed by an OAuth provider, as well as consume a web API that interfaced with the internal legacy business system.

Another challenge for the project was that while the organisation has a strong technical capability focused around its core strength and systems, the project team assembled for this development project were experts in their line of business but new to application system development. To help the project succeed Katalyst chose to "lean in" and provided significant support to the project team regarding technical questions and solutions and when working with IT specialisations within their own own organisation.

In a departure from typical web app designs, we opted for a "stateless" server implementation, with no local or "cloud" database and no local temporary storage. This meant we couldn't employ the usual HTTP pattern of storing a short identifier in a cookie that would be used to look up a more extensive user and session state data object. 

Instead, we utilised client-side cookie storage to hold all the necessary data for the web app server to execute its role. This data included authentication and authorization tokens from the authentication system, essential user account information for the web UI, and business account details related to the user account. All this data was stored securely in a large encrypted cookie that was sent between the client and server as part of every request/response pair.

Managing the size of the resulting cookie was crucial since it might contain multiple JWTs, user account data, and potentially data related to several related business accounts. 

To address this, we employed various techniques:

Upon receiving a request back from the client, the process was reversed: Rails decrypted the data, and our wrapper would reassemble the chunk, decompress the content, and reinflate JSON keys. This resulted in the remainder of the application being able to use a relatively normal design, having access to a comprehensive "User State" which included the data needed for internal API communications (JWT tokens), UI element permission checks, and user-related data for HTML rendering.

Another unexpected issue arose as part of this work related to the large cookie payloads: our standard application stack uses the Nginx web server in front of the Rails engine for HTTP request termination and serving static content without incurring the overhead of the entire web app stack. However, following implementation of our large-cookie approach we would sometimes see request or response failures with only unclear error messages in the Nginx log.

After investigating we discovered that the default size of the total size of headers in incoming requests and outgoing responses were limited to 4KB in the defalt Nginx configuration. To get around this we added the following config directives:

  # allow large response headers
  proxy_buffers 8 16k;
  proxy_buffer_size 16k;
  proxy_busy_buffers_size 16k;

  # allow large request headers
  large_client_header_buffers 8 16k;
  http2_max_field_size 16k;
  http2_max_header_size 16k;

By leveraging client-side cookie storage and careful design decisions, we achieved seamless integration with existing systems despite avoiding a component that would normally be a fundamental part of the web application design.  

Building a database-less Rails application, despite its trade-offs in implementation complexity and functionality, proved to be a very cost-effective solution for our client when ongoing operational costs were considered. 

This approach showcases the power of innovative thinking and strategic design in web app development, and we were very happy that we were able to deliver a solution that delighted the client with both its functionality and its cost effectiveness.