Challenges of big data security governance
To effectively implement data security governance, you must understand key issues such as which assets need protection, the risks they face, whether your organization is compliant, how to handle internal and external attacks, and how to implement security operations. At the same time, big data systems present many security governance challenges due to their characteristics related to storage, users, entry points, data forwarding, and delivery.
Key questions for data security governance
Your ability to answer the following questions clearly and accurately demonstrates how effectively you can implement a security governance project.
What assets need protection?
What risks do these assets face?
Is the organization compliant?
How to handle internal and external attacks?
How should security operations be implemented?
Characteristics and security governance challenges of big data systems
Answering the questions above is difficult because of the unique characteristics of big data systems in terms of storage, users, entry points, data forwarding, and delivery.
Storage
Big data systems are known for having diverse data types (structured, unstructured, and semi-structured) and large data volumes, often at the petabyte (PB) scale. Some large organizations can add hundreds of thousands or even millions of new tables in a single day. This scale poses an enormous challenge for data classification. Manual data classification is unrealistic and prone to omissions.
Users
Big data systems have a large user base that includes all data-related roles. Common users of big data systems include developers, operations staff, and analysts. Even sales and HR staff query the data they need. This wide variety of user types complicates authorization and management. What permissions should be granted to which personnel? What happens if they leave the company or change departments? This can easily lead to permission creep, excessive authorization, and incomplete permission revocation upon departure. All of these issues create significant data security risks.
Entry points
Because big data systems must serve different roles with varying technical skills, they need to provide different entry points for different types of personnel. For example, technical staff can use the command line, but data analysts or operations staff need to use a visual interface or a Business Intelligence (BI) tool.
Different entry points may have different authentication methods, logic, and audit capabilities. A common issue is the transitive trust risk, where multiple users access or operate data from an entry point using a single identity. In addition, because the audit capabilities of upper-layer systems vary, compliance issues such as missing audit events, missing audit messages, or the inability to audit individual users can occur.
Data forwarding
A big data system is typically an end-to-end suite of data development and governance services. It not only collects and processes data but also provides data to business units. Therefore, it has complex data forwarding paths. These paths include but are not limited to ad hoc query paths, offline transmission paths, real-time transmission paths, DataService Studio API paths, and other underlying API or software development kit (SDK) data export channels.
These paths are direct channels for data confidentiality breaches. Data security events can occur if data forwarding security policies are unclear or undefined, underlying paths are not disabled, API security protections are missing, or personnel behavior is not subject to risk control. Examples include illegal cross-border (or cross-domain) data transfer, database exfiltration, and data leaks.
Delivery
Big data systems are responsible for producing data daily for business decisions. The timeliness and accuracy of this data can directly affect the decisions of senior management. Failure to produce timely and accurate data compromises the integrity and availability of the information.
Workflows in big data systems often involve data from multiple departments, owners, and systems. Coordinating these business systems to produce timely and high-quality data, while avoiding production delays or dirty data caused by system breakdowns or invalid input data, is crucial for business continuity and can even affect the trust of senior management.