Nulls and the Empty Sequence

• 28 November, 2022
• By Dave Cassel
• No Comments

We recently came across a neat little gotcha that I thought was worth sharing. I’ve written before about how JSON Nodes and JS objects look the same but act differently in Progress MarkLogic; this is similar but I’m looking at null and the empty Sequence.

Let’s create a really simple document to play with:

'use strict';
declareUpdate();
xdmp.documentInsert(
  "/content/book1.json",
  {
    "book": {
      "title": "A book title",
      "subtitle": "A subtitle"
    }
  }
)

And a template to create a very simple view:

'use strict';

const tde = require("/MarkLogic/tde.xqy");

let template = xdmp.toJSON({
  "template": {
    "context": "/book",
    "rows": [
      {
        "schemaName": "fourV",
        "viewName": "book",
        "columns": [
          {
            "name": "title",
            "scalarType": "string",
            "val": "title"
          },
          {
            "name": "stuff",
            "scalarType": "string",
            "val": "foo",
            "nullable": true
          }
        ]
      }
    ]
  }
});

tde.templateInsert("/tde/book.json", template)

You’ll see that the stuff column looks for the "foo" property, which doesn’t exist. That’s okay, the column is nullable. Now we can do a simple Optic query:

'use strict';

const op = require("/MarkLogic/optic");

op.fromView("fourV", "book")
  .result()

That gives us a Sequence of 1 item that looks like this:

{
  "fourV.book.title": "A book title", 
  "fourV.book.stuff": null
}

As expected we get the title and a null value. Simple as that, right? Well, there’s a wrinkle. That null turns out not to be a null:

'use strict';

const op = require("/MarkLogic/optic");

op.fromView("fourV", "book")
  .result()
  .toArray()[0]["fourV.book.stuff"] === null

That query returns false. Although it gets serialized as null, the actual data structure is an empty Sequence (this is how it gets represented in the index). Instead of comparing to null, we use fn.empty or fn.exists to determine whether there is an interesting value there.

'use strict';

const op = require("/MarkLogic/optic");
const EMPTY_SEQUENCE = Sequence.from([]);
fn.empty(op.fromView("fourV", "book")
  .result()
  .toArray()[0]["fourV.book.stuff"])

Just for fun, what if you really wanted to use the === operator for your comparison instead of fn.empty/exists? You can create an empty Sequence object using Sequence.from.

'use strict';

const op = require("/MarkLogic/optic");
const EMPTY_SEQUENCE = Sequence.from([]);
op.fromView("fourV", "book")
  .result()
  .toArray()[0]["fourV.book.stuff"] === EMPTY_SEQUENCE

That works if you're okay with post-processing. What if you really want to get a where clause in there?

'use strict';

const op = require("/MarkLogic/optic");

op.fromView("fourV", "book")
  .bind(op.as('hasStuff', op.fn.exists(op.col('stuff'))))
  .where(op.eq(op.col('hasStuff'), false))
  .result()

The null values aren't really good for comparison, but we can add something that is. The bind creates a new column. Note that we're calling op.fn.exists to populate it. This is a special form we can use in this way; fn.exists will be called for each row.

The key takeaway is to remain aware of the data type you’re working with. Every now and then its serialization may fool you, but now you’ve got another trick to figure it out.

Share this post:

We recently came across a neat little gotcha that I thought was worth sharing. I’ve written before about how JSON...