Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
Using DataOps and open source together, organizations can reimagine the way they deliver data and software-driven projects
Real-time data is the fuel that powers today’s most critical systems and applications, as well as the digital transformation initiatives organizations need to compete effectively. For the people on the front lines building them, having the right technology alone is not always enough to succeed. To make the most of these game-changing technologies, you also need the right tools and the right strategy in place.
If you’re not maximizing use of real-time data, you’re at risk of being disrupted by a competitor. For this reason, C-level leaders often adopt cutting-edge open source technologies that come with risks, making real-time data applications complex to build, deploy, operate and govern, and require a major investment in time and resources upfront to unlock their potential. You won’t get the most out of open source unless they can be operated and data can be accessed by a wider range of people outside engineering.
For example, open source data technology is often lacking, and documentation is sometimes sparse at best. You may need to ask your data and platform engineers to spend weeks or even months building custom management tools before you can get to work on your first streaming application. That kind of delay can seriously impact your time to market.
How Secure is Your Solution?
Streaming data projects with multiple open source components can also introduce security and compliance issues into the picture. Even if you’re working with the very latest technology, open source often lacks built-in capabilities for security and compliance—especially if they are industry-specific for use cases such as health care or financial services. In fact, a recent survey by Veracode found that 70% of applications have open source security flaws, and 47% of those flawed libraries end up in code indirectly. If your project suffers from increased risk due to security and compliance issues, it can also lead to additional delays in projects, further slowing your time to market.
In a world where the threat landscape is constantly evolving, your organization may wish to opt for a commercial offering for some types of use cases, especially if it includes the features you need to align with very specific compliance and governance needs.
Creating a Culture of Engagement
Arming IT professionals with the right tools and technologies they need is essential, but organizations also need to be sure they have a strategy in place to retain the best people—and keep them productive and satisfied. Fostering a workplace with the right level of engagement can provide a strong start. A recent Forbes article cited studies finding that highly engaged employees result in 21% greater profitability. On the other hand, the same article reported that 67% of employees reported feeling burned out at work—and were 2.6 times as likely to be looking for a different job.
Open source tools are a great way to attract talented engineers because people love working with the latest technology. But as you give your IT professionals access to bleeding-edge capabilities, the complexities and time spent working with them increase. The right tooling can help manage the shortcomings of these technologies and help retain the talent you need by encouraging a positive work-life balance and lead to more successful implementations. If your engineering team isn’t spending time fighting fires and consumed by open source technology, there’s more time to focus on bigger picture issues such as a well-defined career path that keeps team members engaged and more valuable to the business in achieving its goals. With the right level of work-life balance and keeping employees involved and satisfied on the job, you can be more confident that your best engineers will become more than technologists and are strategic business partners who won’t be spending their spare time dusting off their resumes.
Getting the Best of Commercial and Open Source
How can you enable IT professionals with the tools they need to deliver their projects on time and on budget while empowering them with a more strategic role? Employing a DataOps approach to your initiatives can provide a strong start. Spanning many disciplines and skills across the organization, DataOps encourages an environment where people can collaborate, reduce process complexity and fully optimize the use of their data. DataOps allows teams outside of IT and engineering to operate their data directly, without being dependent on a small engineering team. It enables business stakeholders in your workforce to get closer to data and bring their own knowledge and context to data initiatives and improve business outcomes.
For example, leading social and mobile gaming company Playtika needed to provide better visibility data streaming across Apache Kafka, its primary platform for the exchange of information between its microservices. Using a DataOps approach and platform, the company provided its analysts, product owners, operations and developers access to a fully audited GUI and CLI to explore and operate data in a compliant way without relying on data or infrastructure engineering. Providing direct access to data enables 600 developers to save 30 minutes a day as they work on delivering new features—which led to faster and more innovation when it came to developing new games.
When utilized together with a commercial solution based on open source roots, DataOps helps you get your business and technical people on the same page, so your organization can achieve closer alignment to strategic business objectives, with data serving the business. At the same time, you can free up technical staff and developers from tasks such as tooling, and enable them to utilize open source for R&D and other innovative priorities.
Using a collaborative DataOps strategy, your organization can reimagine the way it delivers data and software-driven projects, while also embracing new open source technologies.
