Spring Boot + PostgreSQL + Custom Type = exception!

I do have demo application that shows features of various ORM frameworks. Application performs several basic operation on a database. To have clean database for each application's run I wanted to recreate whole database at its start. This means dropping whole schema and then re-initialising it by executing following script.

DROP SCHEMA IF EXISTS public CASCADE;
CREATE SCHEMA public;

CREATE TYPE plane_dimensions AS (
  length_meters   DECIMAL,
  wingspan_meters DECIMAL,
  height_meters   DECIMAL
);

CREATE TABLE planes (
  id           BIGSERIAL PRIMARY KEY,
  name         VARCHAR(250)     NOT NULL,
  dimensions   plane_dimensions NOT NULL
);

...

Unfortunately this doesn't work.

When the application connects to the database it loads existing types and stores it to a cache. Then my type is dropped and immediately re-created by the script. But re-created type does have different OID than the former (cached). So if you try to use it from the application you will get an exception.

Caused by: org.postgresql.util.PSQLException: ERROR: cache lookup failed for type 1234567

Solution is to re-initialise connection or not to modify the type.

Query to find type's OID:

SELECT oid FROM pg_type WHERE typname = 'plane_dimensions';

Shared test sources in Gradle multi-module project

Basic assumptions

Let's suppose following project structure.

:root-module
|    +--- main source set
|    \--- test source set
\--- :module1
     |    +--- main source set
     |    \--- test source set
     \--- :core-module
              +--- main source set
              \--- test source set

Shared test-utility classes shouldn't be placed in test source set

In some legacy projects I've seen utility classes to be located in test source set. This is problematic in multi-project applications because Gradle does not pull test sources to a dependent module. So module1 won't "see" classes from core-module's test source set. This makes sense because you don't want to have module1's test-classpath flooded with core-modules's test classes.

Note: IDEA 14 does not respect this rule and it pull test sources to a dependent module. This behaviour has been fixed lately.

Test-utility classes shouldn't be placed in main source set

For obvious reasons you don't want to pollute main source set with test classes. In main source set there should be production code only. Tests are usually executed on application's build phase so there is no need to put them into final build.

In Spring application test classes shouldn't be located in component-scanned package

This is mandatory for Spring developers who uses component scan. In some cases you may want to create annotated class used exclusively by tests.

IDEA places all source sets marked as sources to classpath. So if you place your test-only annotated class to a package scanned by Spring your application context will ends up polluted.

Shared test classes should be kept in module where it belongs

Because of previous point some people have tendency to put shared test classes to a stand-alone module. Although core-module-test (main) -> core-module (main), core-module (test) -> core-module-test (main) is legal construct in Gradle you don't want to complicate your project's dependency structure by creating "unnecessary" modules.

Module dependency resolution basics in Gradle and IDEA 14 and 16

Gradle

Gradle allows you to define dependency between two module configurations.

// Declared in root-module
dependencies {
    // Current module's testCompile configuration depends on another module's default configuration.
    testCompile project(':module1')
    // Current module's configuration depends on another module's test configuration.
    testCompile project(path: ':core-module', configuration: 'test')
}

Dependency to a default configuration of another module pulls module's dependencies and sources to a dependent module. Dependency to a test configuration pulls only dependencies without sources. So in our case root-module won't see any core-module's test classes.

IDEA 14

In contrast to Gradle the IDEA 14 pulls all sources to a dependent module. Even test sources.

If you open Gradle project IDEA will behave oddly. It will let you run tests that Gradle won't even compile. Unfortunately IDEA 14 doesn't have any feature that can be used to fix this so you have to think about it when you will write your test.

IDEA 16

Latest version of IDEA does not pull test sources to a dependent module which is consistent behaviour to Gradle. It also allows you to define dependency to a specific source set. You just have to choose create separate module per source set option when importing Gradle project. It has been added lately to harmonize IDEA's dependency resolution with Gradle.

Using shared test sources by dependent module

Declaring dependency to a module's output directory

In projects having utility test-classes in test source set there is usual hot-fix to add testCompile dependency to other module's output directory. So in our example the module1's tests will depend on core-module's compiled test classes.

// Declared in module1
dependencies {
  testCompile project(':core-module').sourceSets.test.output
}

This is quick solution but it has some cons:

• In your IDE you have to rebuild shared classes on every change to refresh dependencies. Remember IDEA 16 does not pull test sources.
• It's against the requirements in first chapter of this article.

Placing utility classes to a custom source set

The good solution is to create custom source set for shared test-classes and declare dependencies to this source set from dependent modules. This approach allows you to keep classes in modules where it belongs still separated from main and test sources. This approach is used by Gradle project itself.

Implementation is quite straightforward. First create script plugin that will add new testFixtures source set and appropriate configurations to a module.

Then apply script plugin on a module and declare necessary dependencies.

// Declared in core-module
apply from: 'testFixtures.gradle'

dependencies {
  testFixturesCompile('junit:junit:4.12')
}

Finally add dependency to the newly created source set.

// Declared in module1
dependencies {
    testCompile project(path: ':core-module', configuration: 'testFixturesUsageCompile')
}

Note for IDEA users: There is a small drawback. If you open the project without create separate module per source set option your custom-source set will be available in main source set. If you use a shared test-class in main source set IDEA won't complain about it but Gradle won't be able to compile. So you have to be careful about what you are using.

Zero downtime deployment with Nginx proxy

At Factorify we wanted to make deployent of our application unnoticed by users. Application is AngularJS client connected to Spring backend. User's requests are proxied by Nginx running on FreeBSD.

Basic idea was to hold user's requests during application deployment. Nginx does not provide such function by default but it is possible to extend it by scripts written in Lua language.

I wasn't able to compile Nginx with Lua support. Fortunately there is Nginx "distribution" called OpenResty which integrates Lua compiler.

Nginx configuration

1. Nginx listens on port 8080.
2. Request is sent to primary node. If primary node is alive response will be served to user.
3. If primary node fails to return response request will be send to backup node.
4. Backup node suspends request for 10 seconds and then forwards it to the backend. 10 seconds should be sufficient time for application to restart. If no response is returned by backend in 10 seconds then error will be send to user.

http {
    upstream backend {
        # 2) Primary node.
        server localhost:8667;
        # 3) Backup node.
        server localhost:8666 backup;
    }

    server {
        # 1) Endpoint exposed to users.
        listen 8080;

        location / {
            proxy_pass http://backend;
        }
    }

    server {
        listen 8666;

        location / {
            # 4) Suspend request for 10 second.
            access_by_lua '
                ngx.sleep(10)
            ';
            proxy_pass http://localhost:8080/;
        }
    }

    server {
        # This is primary node that emulates backend application.
        listen 8667;

        location / {
            default_type text/html;
            content_by_lua '
                ngx.say("Hello World!")
            ';
        }
    }
}

To fine-tune the configuration please refer to Nginx documentation.