← journal open source

journal · 2 august 2026

Two patches, merged

Both of these landed in projects I did not write, in codebases far too large to read. One is Java, one is Python. The code in each was small. Everything around the code was the actual work, so that is mostly what I want to write down.

Apache ShardingSphere: teaching a parser about Doris partitions

Project
Apache ShardingSphere, a distributed SQL layer that shards, routes and rewrites queries
Language
Java, ANTLR4 grammar
Change
3 files, 73 lines added, 2 removed
Merged
30 July 2026 into master
Link
apache/shardingsphere #39201

To do anything useful with a SQL statement, ShardingSphere has to understand it first. That understanding comes from ANTLR4 grammar files, one set per SQL dialect, which describe the shape of every statement the engine will accept. If a dialect has syntax the grammar does not describe, the parser simply fails on it. There is no partial credit.

Doris is one of the supported dialects, and its grammar had a gap around partitions. An open issue listed statements that were failing to parse. Three kinds of partition syntax were falling through:

-- multi range: generate many partitions from a range and a step
PARTITION BY RANGE (dt) (
  FROM ("2022-01-01") TO ("2022-01-31") INTERVAL 1 DAY
)

-- fixed range: an explicit half open interval
PARTITION p1 VALUES [("2022-01-01"), ("2022-02-01"))

-- per partition properties after the values clause
PARTITION p1 VALUES LESS THAN ("2022-01-01")
  ("storage_policy" = "cooldown", "replication_num" = "1")

The useful discovery was that none of this was new syntax to the project. Doris already accepted all three forms in ALTER TABLE, and the grammar already had rules for them. They had just never been wired into the CREATE TABLE path. So the fix was less about inventing grammar and more about reusing rules that were already sitting there.

The core of it was splitting the list of partition definitions so an entry could be either an ordinary partition or a multi range expansion:

// before
partitionDefinitions
    : LP_ partitionDefinition (COMMA_ partitionDefinition)* RP_
    ;

// after
partitionDefinitions
    : LP_ partitionDefinitionItem (COMMA_ partitionDefinitionItem)* RP_
    ;

partitionDefinitionItem
    : partitionDefinition | dorisMultiRangePartition
    ;

dorisMultiRangePartition
    : FROM LP_ expr RP_ TO LP_ expr RP_ INTERVAL expr intervalUnit?
    ;

Then partitionDefinition itself gained the fixed range alternative inside its VALUES clause and an optional (LP_ properties RP_) for the per partition property list, again reusing the existing properties rule rather than writing a new one.

what I would not have guessed

ShardingSphere marks dialect specific edits inside shared grammar with // DORIS CHANGED BEGIN and // DORIS ADDED BEGIN comment fences. I only found that by reading the surrounding file. Nothing in the contributing guide mentions it, and a patch that ignored the convention would have been correct and still wrong.

The rest of the diff, 56 of the 73 lines, is test cases: SQL samples plus the expected parse assertions in the project's XML fixture format. That ratio felt about right. A grammar change with no cases proves nothing, since the whole claim is that a statement that used to fail now parses into a specific tree.

Before opening the PR I ran the Doris parser integration suite locally and put the result in the description: 1261 tests, zero failures. The issue also listed a few SQL fragments that were not actually valid standalone statements, plus one case that already parsed on master. I said so explicitly rather than quietly skipping them, because a reviewer seeing four assertions against a list of seven cases will otherwise wonder what happened to the other three.

Lightly: moving a test suite off unittest

Project
Lightly, a Python library for self supervised learning on images
Language
Python, pytest
Change
7 files, 248 lines added, 301 removed
Merged
28 July 2026 into master
Link
lightly-ai/lightly #2001

This one came from an umbrella issue: the maintainers wanted the whole test suite moved from unittest.TestCase to plain pytest, split into groups so several people could work in parallel without colliding. I took the models group, seven files.

Most of it is mechanical. Drop the TestCase base class, turn setUp into an autouse fixture, replace self.assertEqual with a bare assert, replace assertRaises with pytest.raises, and swap @unittest.skipUnless for @pytest.mark.skipif.

The one genuinely interesting pattern was how the suite handled GPU tests. Every device sensitive test existed twice: a real test defaulting to CPU, and a thin CUDA wrapper that called it back with different arguments.

# before: two methods, one of them a shim
def test_single_projection_head(self, device: str = "cpu", seed: int = 0) -> None:
    ...

@unittest.skipUnless(torch.cuda.is_available(), "skip")
def test_single_projection_head_cuda(self, seed: int = 0) -> None:
    self.test_single_projection_head(device="cuda", seed=seed)

# after: one method, two parametrized runs
@pytest.mark.parametrize("device", ["cpu", "cuda"])
def test_single_projection_head(self, device: str) -> None:
    if device == "cuda" and not torch.cuda.is_available():
        pytest.skip("CUDA not available")
    ...

That is where most of the 301 deleted lines went. The nested subTest loops came out too, since pytest reports a failing parametrized case clearly enough on its own.

the constraint that shapes everything

A refactor PR has exactly one promise: no behavior changes. Which means every judgment call resolves the same way. When a test looked wrong, I left it wrong. When a name was bad, I kept it. Anything I improved along the way would have been a thing the reviewer had to verify separately, and it would have made a mechanical diff into an argument.

There was also an existing config detail worth respecting: five of the seven files are excluded from mypy, and two are not. I left the exclusions exactly as they were, matched what the sibling PRs in the same umbrella issue had done, and confirmed in the description that ruff, mypy and pytest were all clean on the directory.

What actually mattered

Both of these were small diffs, and in both cases writing the code was the short part of the work. The pattern that keeps repeating: