We don’t talk enough about why our work projects failed, and what decisions were made. The invisible censorship committee in our minds get in the way. But failed projects are some of the most interesting to document.
In a series of blog posts, I thought I’d share a few interesting work projects (not necessarily my own) that failed, and offer my best guess as to why. These are personal opinions.
Periscope (now Sisense) is a commercial web-based, sql-driven data visualization tool. Plug in your db credentials, and start writing dashboards straight from sql.
We adopted the tool some time before 2015 and started using the product as an internal tool for analytics. It was an incredibly powerful tool in the early days of a company’s data journey1. The value proposition was: allow non-technical folks to directly ask questions of our data without taking valuable engineering cycles.
The tool was a wild success, and took on use cases like:
It might not be apparent from words, but this is a startling list of features for a visualization tool to support. Each of these could be a separate startup idea for an internal tool.
Our implementation took too many use cases. At scale, it ran operations for the entire company, but suffered from issues with consistency, performance, and security2:
The tool eventually became a source of tech debt. As it accumulated more use cases, dashboards required hacks and tweaks to satisfy those use cases. We also deliberately chose to freeze hiring, which made it difficult to hire folks in to tackle this tech debt.
Eventually, the staffing decision and continued usage lead to insurmountable tech debt.
In some ways, Periscope was a victim of its own success. It took on too many use cases and offered few alternatives to move off. I suspect key features would have helped a lot. One key feature idea is: what if there existed a way to convert a tabular dashboard into a scalable JS prototype?
We eventually decided that direct SQL access was an anti-pattern and found another tool which promised to help defend against tech debt. In reality, tech debt creeps in on anytime you’re making rapid changes.
How do we do better? If you’re in the same position as us, you’ll have to redesign your data architecture and re-align teams. Hopefully if your engineering teams aren’t too big, you can take small, incremental steps to get there (coming in a future blog post).
If you’re at the stage right before you move onto the cloud, you can mitigate most of this by shifting to cloud solutions and refactoring your debt. I’ve seen this done many different ways. If you’re curious, reach out. I’m always happy to chat about this.
1Internal tools are particularly powerful at our company because we have
2Note these were the issue as of 2015. The current version may be much better