CFOtech Asia logo
Technology news for Asia's CFOs and financial decision-makers
Partner content
Story image

The winning formula for successful DataOps

By Contributor
Tue 28 Jul 2020
FYI, this story is more than a year old

Article by Qlik APAC vice president of data integration, Byron Yeung.

When the COVID-19 crisis hit earlier this year, organisations were forced to quickly reinvent how they operate. Today, many businesses have accelerated their digital transformation and quickly embraced new technologies.

As a result, IT demand has grown across areas such as cloud and data analytics and reduced across legacy categories. This makes it an ideal time for IT teams to streamline the development and delivery of data to better support business operations.

Enter DataOps. Similar to DevOps, DataOps is an emerging set of practices, processes and technologies for building and enhancing data analytics pipelines to better meet the needs of the business.

The method promises improved productivity, streamlined and automated processes, increased data output and greater collaboration across teams. Ultimately, DataOps accelerates time to insight and solves the many challenges associated with data availability. So, what does it look like in practice?

Establishing the right foundation

While data managers, architects and engineers are embracing new cloud and data lake initiatives to provide a more scalable and agile infrastructure, they need to avoid the mistakes of first-generation big data projects that fail to deliver real business value and become large landing zones for corporate data.

Agile organisations require modern data architectures that allow for the quick use of the ever-expanding volumes of data. These infrastructures – based on hybrid and multi-cloud for greater efficiency – provide enterprises with the flexibility they need to compete more effectively, improve customer satisfaction and increase operational efficiencies.

When the DataOps methodology is part of these architectures, companies are empowered to support real-time data analytics and collaborative data management approaches while easing the many frustrations associated with access to analytics-ready data.

Adding the key ingredients for DataOps success

Before shifting to modern analytics and data management platforms in the cloud, organisations should first evaluate their legacy integration technology to ensure it can support the key DataOps principles that will accelerate time to insight.

Here are four considerations to ensure DataOps success:

Maintain continuous integration 

Continuous integration is foundational to modern data platforms and to meet the needs of real-time analytics. It requires individuals to think differently about integration, it can no longer be a batch process that only happens at certain points in time and impact transactional systems.

Applying technologies such as change data capture (CDC) provides a non-invasive method to catch data and metadata changes from transactional systems, relational databases and applications and stream them in real-time to the analytic platform.

Ensure solutions are universally applicable

Seek solutions that support a broad variety of sources and targets and span multiple integration use cases. For example, you should only capture changed data and route that data where and when it’s needed: to a data lake; or a data warehouse; or replicate it to another database; etc. This provides greater efficiency and fewer moving parts.

Automate your processes

It’s always a challenge to find the right people and skill sets to meet the everchanging technology landscape. Automation becomes essential to the success of a DataOps initiative, from the generation of change data streams to delivery to refinement and finally the creation of analytics-ready data sets.

This becomes particularly important for data lakes where heterogeneous data is coming in a variety of formats and is continuously updated with new change files.

Apply an agile approach 

Being able to embrace and adopt new technologies and implement new data pipelines is vital due to the continuous change in data formats, platforms, technology and data loads. An enterprise must be able to quickly pivot or it can be left behind the competition.

Flexible integration solutions allow for IT to simply change a source or target without disrupting the entire infrastructure – providing an agile, modern infrastructure that future proofs an enterprise.

With COVID-19 forcing organisations to embrace transformation in order to continue operating, having the right information and analysis could mean the difference between leading the market or ceasing to exist.

This is further shown in a new global study conducted by IDC, where a clear association was identified amongst companies that were more capable of turning data into insights for decision-making. In fact, the Australian business leaders surveyed pointed to a significant gain to the bottom line from investing in a strong data pipeline.

DataOps is improving the way that data is being made available across the enterprise and is the key to better business insights and a competitive edge. 

Related stories
Top stories
Story image
Storage
EXCLUSIVE: Finding the best data center for your business needs with datacenterHawk
Companies using cloud are consistently looking for the best storage solutions to suit their enterprise needs and often have to go through rather complex processes in order to find the right fit.
Story image
Ransomware
Examining the future of ransomware threats with Vectra’s CTO
As customers' valuable data move to the cloud, so will ransomware. What is the current landscape and what do we need to know?
Story image
Recruitment
Thales on recruitment hunt for next disruptive innovations
"Recruiting new talent is part of Thales's belief in the power of innovation and technological progress to build a safer, greener and more inclusive world."
Story image
Cloud
BT builds on Equinix partnership with new cloud offering
BT has launched a next-generation cloud connectivity offering extending its global network into strategic carrier-neutral facilities (CNFs) and building on its existing partnership with Equinix.
Story image
Digital Transformation
What CISOs think about cyber security, visibility and cloud
Seeking to uncover the minds of CISOs and CIOs across Asia Pacific, my company recently asked Frost & Sullivan to take a snapshot of cloud adoption behaviour in the region.
Story image
Open source
DataStax secures US$115 million to fund database expansion
DataStax has secured US$115 million in funding, which it will use to develop and expand its Astra DB multi-cloud database and Astra Streaming service globally.
Story image
Public Cloud
Public cloud services revenues top $400 billion in 2021
"For the next several years, leading cloud providers will play a critical role in helping enterprises navigate the current storms of disruption."
Story image
Partnerships
Thailand announces launch of the Thailand 5G Alliance
It will promote collaboration between the public and private sector, through companies such as Huawei, to commercially drive Thailand's 5G development.
Story image
Testing
Video: 10 Minute IT Jams - An update from Tricentis
Tricentis provides software testing automation, and software quality assurance products for enterprise software.
Story image
Multi Cloud
Cloud is a tool, not a destination
For many years, “cloud” has been thought of as a destination which has led to a misguided strategy that sees an enterprise trying to shift all its applications to a single cloud provider – regardless of the specific needs and nuances of each individual workload.
Story image
Supply chain
Supply chains continue to be disrupted, enterprises embrace circular economy
“Businesses urgently need to find a solution that can help them to manage this disruption, and transition to a circular economy."
Story image
DNS
DigiCert acquires DNS Made Easy and affiliated brands
Greg Clark comments, says, "This combination enhances the security of certificate validation and enables the automation of future validations."
Story image
Data ownership
Brands must reclaim trust by empowering data ownership
According to Twilio's new State of Personalisation Report 2022, 62% of consumers expect personalisation from brands, and yet only 40% trust brands to use their data responsibly and keep it safe.
Story image
Artificial Intelligence
Juniper study reveals top AI trends in APAC region
Juniper's research shows an increase in enterprise artificial intelligence adoption over the last 12 months is yielding tangible benefits to organisations.
Story image
Cybersecurity
Vulnerable APIs costing businesses billions every year
Large companies are particularly vulnerable to the security risks associated with exposed or unprotected APIs as they accelerate digital transformation.  
Story image
Artificial Intelligence
Siemens expands NVIDIA partnership for industrial metaverse
Siemens is expanding its partnership with NVIDIA to enable the industrial metaverse and increase the use of AI-driven digital twin technology.
Story image
Broadband
Singapore found to have the speediest internet rates in the world
New research from BanklessTimes has shown that Singapore has the highest recorded median internet speed in the world at 207.61 MBPS.
Story image
Sustainability
Honeywell launches new carbon energy management software for buildings
The new Carbon & Energy Management service allows building owners to track and optimise energy performance against carbon reduction goals, down to a device or asset level.
Story image
Education
Microsoft Hong Kong partners with CUHK Business School
The collaboration sees CUHK Business School become the first business school as the authorised exam centre for Microsoft Certifications in Hong Kong.
Story image
Data
Kacific and Teleglobal support expansion of telecom networks in Indonesia
Kacific has partnered with Teleglobal in Indonesia to provide a large-scale deployment of mobile backhaul services to major telecommunications operators.
Story image
Infrastructure
Global investment in data centers more than doubled in 2021
DLA Piper's latest global survey finds the total investment in data center infrastructure worldwide rose from USD $24.4 billion in 2020 to USD $53.8 billion in 2021.
Story image
Cloud
Cloudflare outage in 19 data centers worldwide due to own error
Cloudflare says its outage for 19 of its data centers yesterday was because of a change in a long-running project to increase resilience in its busiest locations.
Story image
Music
Mastercard reveals first-ever album titled Priceless
Mastercard's music album Priceless has been unveiled at the Cannes Lions Festival of Creativity and features 10 songs by 10 artists worldwide.
Story image
Infosys
Preparing for the digital decade with the right workforce strategies
For a decade that started under the pall of the pandemic, the 2020s is poised to end with a bang with the digital economy swelling to a high across the world.
Story image
Tech job moves
Tech job moves - ActiveCampaign, Arcserve, LogRhythm & Qlik
We round up all job appointments from June 17-22, 2022, in one place to keep you updated with the latest from across the tech industries.
Story image
Tech job moves
Tech job moves - Bitdefender, Cohesity, Fortinet & MODIFI
We round up all job appointments from June 27-30, 2022, in one place to keep you updated with the latest from across the tech industries.
Story image
Digital wallets
NFTs are ready to disrupt the ticketing world
The last few months have seen NFTs wielded by digital creators to take ownership over their craft and content. Now other industries are beginning to understand the real-world value that these nifty decentralised tokens can provide.
Story image
Airwallex
How Airwallex helps businesses achieve globalisation success
As markets continue to shift, businesses need to be able to provide the same quality of service for customers regardless of where they are located around the world.
Story image
Market growth
Salesforce unveils new offerings for consumer goods companies
Salesforce has announced new products for consumer goods companies to help brands navigate increasing market complexity more easily.
Story image
Gartner
Gartner's top recommendations for security leaders
"Leaders now recognise that major disruption is only one crisis away. We can’t control it, but we can evolve our thinking, philosophy, program and architecture.”
Story image
Cybersecurity
Video: 10 Minute IT Jams - An update from CrowdStrike
Scott Jarkoff joins us today to discuss current trends in the cyber threat landscape, and the reporting work CrowdStrike is doing to prevent further cyber harm.
Story image
Cryptocurrency
NOWPayments launches new service to analyse cryptocurrency fees
NOWPayments has launched a new network fee optimisation solution that analyses current network fees and picks the most profitable option out of the client's payout wallets.
Story image
Talend
Forrester names Talend Leader in enterprise data fabric
Forrester has named Talend a leader among enterprise data fabric providers in the Forrester Wave: Enterprise Data Fabric, Q2 2022 report.
Story image
Contact Centre
Customer service agents don't want to return to contact centres
A new report has revealed that 85% of customer service agents want to work full-time at home and not return to contact centre offices.
Story image
Infrastructure
New VMware offerings improve cloud infrastructure management
VMware has unveiled VMware vSphere+ and VMware vSAN+ to help organisations bring benefits of the cloud to existing on-prem infrastructure.
Story image
Amazon Web Services / AWS
Qualtrics goes live on AWS Cloud Infrastructure in Japan
Organisations across Japan will now be able to access the Qualtrics XM/OS platform locally via data centre in the AWS Asia Pacific (Tokyo) region.
Story image
Artificial Intelligence
Accenture shares the benefits of supply chain visibility
It's clear that gaining better visibility into the supply chain will help organisations avoid excess costs, inefficiencies, and complexity to ultimately improve their bottom line.
Story image
State Library of Victoria
State Library of Victoria entrusts Oracle support and security to Rimini Street
“Our finance team are very happy with the support and security that Rimini Street provides, which keeps our assets and our customers secure."
Story image
Cybersecurity
Tech and data’s role in the changing face of compliance
Accenture's study found that 93% of respondents agree or strongly agree new technologies such as AI and cloud make compliance easier.
Story image
Collaboration
IT and security team collaboration crucial to data security
Many IT and security decision makers are not collaborating as effectively as possible to address growing cyber threats.
Story image
Artificial Intelligence
Salesforce harnesses automated solutions with new developments
Salesforce has launched Sales Cloud Unlimited, a new feature to help accelerate productivity with AI and automation.
Story image
API
Industry-first comprehensive risk-based API security enhances protection
Application Programming Interfaces (APIs) have become a crucial part of operating web and mobile application businesses and are causing significant economic growth in the digital sector.
Story image
BitTitan
How to achieve your monthly recurring revenue goals
Monthly recurring revenue (MRR) is the ultimate goal, the most important issue on which anyone in the IT channel should focus.
Story image
Cybersecurity
Kaspersky opens three new centers to boost data management
Cybersecurity company Kaspersky has opened three new Transparency Centers, one in Japan, the second in Singapore and the third in the United States.