Global search tool

I’ve written before about the tools we use to run the newspaper business, organize our news coverage, and publish the printed edition of the paper. The number of tools has grown over time, and I recently realized we’d reached the point where finding information across all of our tools had become too time consuming.

“Were those public budget hearing details in a legal notice email message we received, or was that in that one shared Google Doc, or did it just make it straight in to a news story we published, or maybe it was in a PDF we’d tucked away in a folder?” It could take someone many browser tabs and clicks to find out, and that’s the last thing you want to do when trying to put together coverage on a deadline.

To solve this, we partnered with the great folks at Tighten to build a global search tool. It’s an internal Laravel application that allows our team to enter a search query and see results across multiple tools, all in one place:

(Redacted for public sharing, obviously the real results are not.)

The results page is a flexible Livewire component so there’s a toggle to show the entries in a more standard search engine results format:

There’s a simple admin interface to manage and add new sources to search:

Permissions management is mostly controlled through our single sign on authentication system, so we can designate access to a given tool’s search results based on user role.

(Interestingly to perhaps no one but me, one of our core tools had no API endpoint to search all records by keyword, so I ended up on a side quest to build a second Laravel app that just archives all records from that tool in another local searchable database. It periodically queries a “what records have been created or changed in the last few minutes” API endpoint that was available to bring the information in and then turns around and provides a usable search API endpoint.)

This all required a significant investment of time and money to get in place, but for an organization that’s all about finding, organizing and publishing information in useful ways, I think it was also a significant quality of life improvement for the team, and I think it will serve us well over time.

I have thoughts of releasing the software as open source. If anyone’s interested, please let me know.