The last time we discussed filters in depth, back in the Finding Things chapter, we learned that there is such a thing as a filter run, indicated by those outer square brackets, and that you can have more than one filter run in a filter. So what does having more than one run do for us?

Essentially, it lets us build up more complicated logic. With a single run, we can only find tiddlers that match all of a set of conditions. With multiple runs, we can combine AND, OR, and NOT conditions as much as we like, although due to the “pipeline” nature of filters, the logic works a little differently than the conventional Boolean logic used in Excel or a typical programming language.

Creating OR conditions

Let's imagine we want to find all tiddlers which are either meetings or journal entries. If we limit ourselves to a single run, it's easy to find tiddlers which have both tags by using two steps in the run: [tag[Journal]tag[Meeting]]. But the only way we can find ones that have either tag without using multiple runs is to run two filters and manually combine the results in our heads. That's no good at all!

Fortunately, all we have to do is create two runs by putting the filter steps in different outer sets of square brackets:

[tag[Journal]] [tag[Meeting]]

TiddlyWiki will process each of these runs separately, gathering first all the tiddlers that are tagged Journal and then all the tiddlers that are tagged Meeting, and combine the results. Unless we add an additional sort step or the same tiddler has both tags (more on that in a moment), all of the tiddlers tagged Journal will come before all the tiddlers tagged Meeting.

We can, of course, add additional steps to either of the filter runs. For instance, maybe we want to see all journal entries, along with only the meetings that included Jane:

[tag[Journal]] [tag[Meeting]contains:participants[JaneDoe]]

Putting spaces between filter runs is optional but can help to make filters more readable. (Don't put spaces between filter steps, though: as noted in Using Filter Expressions, your filter won't work as expected if you do that.)

Another extremely common OR condition is a simple list of tiddlers. For instance, imagine we want to get tiddlers A, B, and C:

[[A]] [[B]] [[C]]

Or, since the names A, B, and C don't have spaces in them, we can just say:

A B C

You may notice that this is exactly the same format as a tiddler list. This, incidentally, is why, in a note at the end of Filters and Transclusions, we were able to set the filter attribute of a list to simply the {{!!family}} list field and have it work like a filter.

Don't, however, take this to mean that tiddler lists and filters are the same thing. While a tiddler list is a valid filter, most filters are not valid tiddler lists; you can't, for example, use a filter run like [tag[Contact]] as an element of a tiddler list and expect it to work.

Dominant appending

You might wonder what happens if the same tiddler comes up in multiple filter runs. Let's try it. Start with this filter:

[tag[Meeting]]

You should see several meetings. Now try adding tiddlers that are tagged OnboardingHr, since we have at least one meeting that's part of that project:

[tag[Meeting]] [tag[OnboardingHr]]

You should notice several things. First, no duplicate entries were created. But second, the order changed. Why's that?

Think about how TiddlyWiki works within filter runs for a moment. It starts with the leftmost filter step, then passes the output of that step to the next step to the right, which does something with that output and produces its own output, and so on until we reach the end of the filter run.

TiddlyWiki does something similar with filter runs. The output of the first filter run doesn't affect the input of the second filter run – the second filter run gets a fresh set of input tiddlers – but TiddlyWiki does execute each filter run in order from left to right and combine the results as it finishes each. It's the way it combines the results that can get tricky.

We'll call the output of the filter run that's currently being executed the current run and the set of output produced from combining all of the previous filter runs the accumulated results. (The accumulated results can be empty – for instance, if this is the first filter run, or if all previous filter runs returned no results.)

When a run finishes, TiddlyWiki dominantly appends the output from the current run to the accumulated results. During a dominant append, if the item doesn't exist in the list of accumulated results already, it goes at the end of the list, as you would expect. However, if the item already exists in the accumulated results, it is actually removed from the list in its current position and then added back at the end. This is why you saw the order of the list change – an item that wasn't already the last item in the accumulated results was shifted to the end because it also showed up in the second run.

Run prefixes

Dominant appending is the default way to combine multiple runs. But you can change this behavior by placing the character =, +, -, or ~ (or the equivalent named prefixes :all, :and, :except, and :else), directly in front of the second filter run, e.g., [tag[Journal]] =[tag[Meeting]]. These work as follows:

= or :all

Just slap the results of the runs together, rather than performing deduplication by dominantly appending; A B C and B C D results in A B C B C D, rather than the default of A B C D.

+ or :and

A few paragraphs up, we said each filter run gets its own input. That's not quite true; actually, each filter run gets its own input unless you put a + in front of it, in which case its input is the accumulated results and its output replaces everything output so far. You can usually accomplish the same thing by just combining the filter runs together into one run (e.g., [tag[Meeting]] +[tag[Journal]] is an overly complex way to say [tag[Meeting]tag[Journal]]), but sometimes you'll want to do something like OR together the results of several filter runs and then AND their results with another run.

- or :except

Remove items in this run from the accumulated results, if they were present (Boolean NOT condition).

~ or :else

If the list of accumulated results is currently empty, execute this run and make its output the accumulated results. Otherwise, ignore the run completely.

The prefix :or is equivalent to no prefix.

There are a number of other prefixes, added in more recent versions, that are only available in the named variants. They're a little harder to understand, and they're less often necessary. You can skip them for now if you like, and if we need one in the exercises throughout the rest of Grok TiddlyWiki, instructions will be provided in the exercise. But they're still quite useful, so you might want to come back on your own and explore them in the future!

More named prefixes

:cascade

Evaluate this filter run, which should return a series of other filters. Then evaluate those filters in turn on the input, and output the results of the first filter which has a value. This is used with the cascade customization mechanism; we won't cover cascades in detail in Grok TiddlyWiki, but once you've worked your way through the book they should be easy to pick up from the documentation.

:filter

Evaluate the filter run on each tiddler in the accumulated results and output elements where the filter expression does not evaluate to an empty list. This differs from + in that + makes the output of its run the accumulated results. :filter doesn't use the output of its run at all, except to check whether it's non-empty.

:intersection

Like +, but the second filter run is evaluated completely before being ANDed. The TiddlyWiki documentation gives this example: if you are trying to see what tags two tiddlers have in common, you might say [[Tiddler 1]tags[]] +[[Tiddler 2]tags[]]; but this wouldn't work because [[Tiddler 2]] is a constructor and will wipe out anything from the first filter run. :intersection evaluates the two filter runs in isolation, so that the ANDing takes place on the sets of tags once they've been expanded.

:map

Apply this filter run to each item in the accumulated results in turn, and replace each item in the accumulated results with the output of the mapped filter run.

:reduce

Flatten the accumulated results into a single value by applying this filter run to each tiddler in the accumulated results in turn. Within this inner filter run, a variable accumulator contains the single value produced by the previous execution of the filter run. The output of the last input tiddler becomes the run's output.

:sort

Apply this filter run to each tiddler in the accumulated results in turn, and sort the accumulated results by the key produced by this filter expression. That is, the filter run followed by :sort transforms each input into the form needed for sorting. This is the same thing as the sortsub operator, but it's often easier to use because you don't have to define a separate variable containing the sort filter.

:then and :else

These work just like the then and else filter operators, except they apply to an entire run rather than an individual tiddler in the filter pipeline: if there's at least one tiddler in the accumulated results, the accumulated results are replaced with the results of the :then run, and if the accumulated results were empty, so with the :else run.

You can read more about these run prefixes in the documentation.

Exercises

Takeaways

Takeaways are not available in the static version of Grok TiddlyWiki. Visit the wiki version of this page to study takeaways.