Upgrading from Lucee 5 to Lucee 6

Recently we decided to migrate a project I work on from Lucee 5.3 to Lucee 6.0 and we ran into some issues, so I thought I would document them here to help others looking to do the same migration.

Why Lucee 5.3?

Firstly, you might be wondering why we were not going from 5.4 to 6.0. This was because before we undertook this migration, we were using a Redis Sentinel cluster for our caching and the Lucee Redis Extension does not support Sentinel (LDEV-4579) from version 3.x so we were still needing to use version 2.9, however, that version doesn’t work with Lucee 5.4 (LDEV-4674). So, the first thing we did was to reconfigure our Redis cluster not to be using Sentinel.

Issues

Before we begin, let me be clear, there is nothing inherently wrong with Lucee 6.0 and it is a good progression for Lucee overall. There are, however, breaking changes, which you should expect going from one major version to another.

Cache race condition

I’m not sure if this is a bug or by design but in Lucee 6 it would appear that the cachePut() function no longer waits for a response from the cache provider (in our case Redis) before moving on. In the past we had code blocks that looked like this:

if ( !cacheKeyExists( some_cachekey ) ) {
	var some_data_to_cache = get_some_data();
	cachePut(some_cachekey, some_data_to_cache, createTimespan(0, 6, 0, 0));
}
var some_data = cacheGet(some_cachekey);
echo(some_data);

However, after upgrading to Lucee 6.0 this would error on the echo() line with an error about the some_data variable not existing. This is because when cacheGet() asks for a key that does not exist, it returns null and that then means the variable you are putting the response into never exists. In Lucee 5.3, the above code worked without issue. However, this was relatively easy to fix in the following way:

if ( !cacheKeyExists( some_cachekey ) ) {
	var some_data = get_some_data();
	cachePut(some_cachekey, some_data_to_cache, createTimespan(0, 6, 0, 0));
} else {
	var some_data = cacheGet(some_cachekey);
}
echo(some_data);

To be honest, this is a better way of doing it as you are not doing both the cachePut() and cacheGet() within the same request.

deserializeJSON

In Lucee 6 the deserializeJSON() function now errors if you pass it an empty string, however, in Lucee 5 it would return an empty string. This was changed for compatibility with Adobe ColdFusion (LDEV-3413) so we had to make a change in a few places where it was possible to sometimes have an empty string passed into the deserializeJSON(). We changed them to be like the following:

deserializeJSON( (some_json == '' ? '{}' : some_json ) );

Caching MySQL queries

This was the issue that caused us the most problems and took us the longest to work around. This one is a bug in Lucee 6 and needs to be resolved. It affected the caching of queries from MySQL with both Redis and memcached.

We started off having issues that we couldn’t explain, it was erroring saying items with a structure that was cached didn’t exist but other items created at the same time within the same structure would exist. We noticed if we dumped the item before putting it into the cache that the MySQL query in the structure existed, but when we retrieved it from the cache, the structure was there but the query item(s) had been removed.

At first, we thought this was a problem with our new Redis setup so we decided to try using a different cache provider, memcached, however, this time memcached gave us an error:

java.lang.RuntimeException:java.io.NotSerializableException: com.mysql.cj.jdbc.result.ResultSetMetaData

So, something had changed in how Lucee was serializing the MySQL queries when sending them to the cache and for MySQL query objects it was erroring saying it couldn’t serialize them. After a lot of trying different things, we figured out that if we passed the query through a filter and essentially changed it from the MySQL query object to just a query object it could then be cached by both Redis and memcached:

testQry = queryExecute(
    sql = "SELECT 1 AS result FROM users LIMIT 1",
    options = { datasource = 'users'}
).filter( (q) => {
    return true
})

After we figured this out and got the code working I raised a ticket with Lucee about the issue LDEV-4871.

datediff() change

After going live with Lucee 6 in production, we had a horrible memory leak that was causing the system to crash periodically. We tracked this back to a process that stores large ZIP files temporarily in memory for 5 minutes and cleared them using a task. However, a change to the functionality of the datediff() for Adobe ColdFusion compatibility (LDEV-2044) meant that instead of looking for files that were 5 minutes or more old, the task was now looking for files that were created 5 minutes in the future and therefore wasn’t clearing anything from memory. Once the memory was exhausted the system would crash. This was a simple code change to fix, but another “gotcha” with the upgrade process.

List of breaking changes

One thing we didn’t find until after we had finished the fixes to the code, was that on the Lucee JIRA, there is a list of all the tickets that cause breaking changes between Lucee 5 and Lucee 6, you can find that list here:

https://luceeserver.atlassian.net/browse/LDEV-4534

Overall it was good to get the upgrade done and keep our system as up-to-date with Lucee as possible, but it did take a lot longer than we were expecting it to, mainly due to the MySQL query caching issue.

• Category: Uncategorized