DAMA - Rocky Mountain Chapter News

Registration now open for DAMA-RMC's 2026 Q1 Event!

Thu, 01 Jan 2026 19:00:00 GMT

Join us for our upcoming Q1 Chapter Event, Data Discussions: AI At Scale, on Thursday, February 5th from 3:00pm to 5:30pm in Greenwood Village. Doors open at 2:00pm for networking!

DMBoK Figure 91 Context Diagram: Data Quality

Tue, 31 Dec 2024 14:00:00 GMT

Data Quality can be defined as the degree to which dimensions of Data Quality meet the requirements. This implies that requirements should be formulated for each (relevant) dimension. A much shorter definition for quality of data is ‘fit for purpose.’

Data that meets the requirements are of sufficient quality; data that doesn’t meet the requirements are of insufficient quality. To keep it simple, we respectively speak of high and low, or poor quality data.

Effective Data Management involves a set of interrelated processes enabling an organization to use its data to achieve strategic goals. An underlying assertion is that the data itself is of high quality. Data Quality Management is the planning, implementation, and control of activities that apply quality management techniques to data in order to assure it is fit for consumption and meets the needs of data consumers.

High quality data is context driven. This means that the same data may be simultaneously viewed as high quality by some areas of an organization while being viewed as low quality by other areas. Many organizations fail to engage with this question of context, that is, high Data Quality being that which is fit for purpose.

If we understand organizations as data manufacturing machines, we can assert (from our experience in manufacturing) that organizations that formally manage the quality of data will be more effective, more efficient and deliver a better experience than those that leave Data Quality to chance. However, no organization has perfect business processes, technical processes, or data management practices. In reality, all organizations experience problems related to their Data Quality. Many factors undermine quality data: lack of understanding about the effects on organizational success, leadership that does not value Data Quality, poor planning, ‘siloed’ system design, inconsistent development processes, incomplete documentation, a lack of standards, or a lack of Data Governance.

As is the case with Data Governance and with Data Management as a whole, Data Quality Management is a function, not a program or project. This is because projects and even programs have starts, middles, and ends. A Data Quality Function is, or should be, a continuing business as usual set of activities. It will include both projects and programs (to address specific Data Quality improvements) as well as operational work, along with a commitment to communications and training. Most importantly, the long-term success of a Data Quality improvement program depends on getting an organization to change its culture and adopt a quality mindset. As stated in The Leader’s Data Manifesto, “fundamental, lasting change requires committed leadership and involvement from people at all levels in an organization.” People who use data to do their jobs – which in most organizations is a very large percentage of employees – need to drive change. One of the most critical changes to focus on is how their organizations manage and improve the quality of their data.

Formal Data Quality Management is similar to continuous quality management for other product manufacturing. It includes managing data through its lifecycle by setting standards, building quality into the processes that create, transform, and store data, and measuring data against standards. Managing data to this level usually requires a Data Quality Function team. The Data Quality Function team is responsible for engaging both business and technical data management professionals and driving the work of applying quality management techniques to data to ensure that data is fit for consumption for a variety of purposes. The team will likely be involved with a series of projects through which they can establish processes and best practices while addressing high priority data issues.

#damarmc TechYeet Slack Channel

Fri, 27 Dec 2024 14:00:00 GMT

DAMA Rocky Mountain Chapter is happy to announce its partnership with TechYeet. A Slack community based on connecting people in the wider data and technology communities. With over 5,000 TechYeet members, DAMA-RMC is utilizing the TechYeet community platform to bring people together in the Data Management space.

Please reach out to Greg Sheridan PMI-ACP, VP of Partnerships & Sponsorships, at PartnershipsVP@damarmc.org, if you are in TechYeet and would like to join the DAMA-RMC channel.

DMBoK Figure 90 Sample System Lineage Flow Diagram

Thu, 26 Dec 2024 14:00:00 GMT

Although a lineage graphic, such as in last week's figure, describes what is happening to a particular data element, not all business users will understand it. Higher levels of lineage (e.g., ‘System Lineage’) summarize movement at the system or application level. Many visualization tools provide zoom-in / zoom-out capability, to show data element lineage in the context of system lineage. For example, this figure shows a sample system lineage, where at a glance, general data movement is understood and visualized at a system or an application level.

As the number of data elements in a system grows, the lineage discovery becomes complex and difficult to manage. In order to successfully achieve the business goals, a strategy for discovering and importing assets into the Metadata repository requires planning and design. Successful lineage discovery needs to account for both business and technical focus:

Business focus: Limit the lineage discovery to data elements prioritized by the business. Start from the target locations and trace back to the source systems where the specific data originates. By limiting the scanned assets to those that move, transfer, or update the selected data elements, this approach will enable business data consumers to understand what is happening to the specific data element as it moves through systems. If coupled with Data Quality measurements, lineage can be used to pinpoint where system design adversely impacts the quality of the data.

Technical focus: Start at the source systems and identify all the immediate consumers, then identify all the subsequent consumers of the first set identified and keep repeating these steps until all systems are identified. Technology users benefit more from the system discovery strategy in order to help answer the various questions about the data. This approach will enable technology and business users to answer question about discovering data elements across the enterprise, like “Where is social security number?” or generate impact reports like “What systems are impacted if the width of a specific column is changed?” This strategy can, however, be complex to manage.

Many data integration tools offer lineage analysis that considers not only the developed population code but the data model and the physical database as well. Some offer business user facing web interfaces to monitor and update definitions. These begin to look like business glossaries.

Documented lineage helps both business and technical people use data. Without it, much time is wasted in investigating anomalies, potential change impacts, or unknown results. Look to implement an integrated impact and lineage tool that can understand all the moving parts involved in the load process as well as end user reporting and analytics. Impact reports outline which components are affected by a potential change expediting and streamlining estimating and maintenance tasks.

Give the Gift of Professional Membership!

Tue, 24 Dec 2024 20:25:39 GMT

Gift a colleague, or yourself, a 25% off discounted DAMA-RMC professional membership this holiday season. Join as a professional member OR upgrade from a guest membership. Professional membership includes:

Entry to all chapter meetings AND mingle events
Meeting and presentation archive access
CDMP virtual study group, bootcamp and discounts
DMBoK discount
Conference discounts
DAMA events and programming discounts
Plus so much more...

Promo Code: 12HOLIDAY25

Join HERE.

Congratulations New CDMPs!

Mon, 23 Dec 2024 20:10:17 GMT

Thanks to everyone who participated in DAMA-RMC's study sessions, bootcamp, and "Pay-If-You Pass" exam prep over the last few months.

We are thrilled at the progress everyone made and excited to announce 6 new CDMPs:

Funke Bishi

John Lieto

Katrina Miyamoto

Kris New

Benjamin Seidle

Rachel Udow

Several others will be completing their tests in the next few weeks.

We wish everyone luck!

Learn more: CDMP Certification with DAMA-RMC Support

Questions?

Please contact ProfessionalDevelopmentVP@damarmc.org.

December 2024 Newsletter

Fri, 20 Dec 2024 14:00:00 GMT

December 2024 Newsletter.pdf

DMBoK Figure 89 Sample Data Element Lineage Flow Diagram

Thu, 19 Dec 2024 00:30:44 GMT

A key benefit of discovering and documenting Metadata about the physical assets is to provide information on how data is transformed as it moves between systems. Many Metadata tools carry information about what is happening to the data within their environments and provide capabilities to view the lineage across the span of the systems or applications they interface. The current version of the lineage based on programming code is referred to as ‘As Implemented Lineage’. In contrast, lineage describe in mapping specification documents is referred to as ‘As Designed Lineage’.

The limitations of a lineage build are based on the coverage of the Metadata management system. Function-specific Metadata repositories or data visualization tools have information about the data lineage within the scope of the environments they interact with but will not provide visibility to what is happening to the data outside their environments.

Metadata management systems import the ‘As Implemented’ lineage from the various tools that can provide this lineage detail and then augment the data lineage with the ‘As Designed’ from the places where the actual implementation details is not extractable. The process of connecting the pieces of the data lineage referred to as stitching. It results in a holistic visualization of the data as it moves from its original locations (official source or system of record) until it lands in its final destination.

This figure shows a sample data element lineage. In reading this, the ‘Total Backorder’ business data element, which is physically implemented as column zz_total, depends on 3 other data elements: ‘Units Cost in Cents’ physically implemented as ‘yy_unit_cost’, ‘Tax in Ship to State’ implemented in ‘yy_tax’ and ‘Back Order Quantity’ implemented in ‘yy_qty’.

DMBoK Figure 88 Example Metadata Repository Metamodel

Wed, 11 Dec 2024 14:00:00 GMT

A Metadata Management system must be capable of extracting Metadata from many sources. Design the architecture to be capable of scanning the various Metadata sources and periodically updating the repository. The system must support the manual updates of Metadata, requests, searches, and lookups of Metadata by various user groups.

A managed Metadata environment should isolate the end user from the various and disparate Metadata sources. The architecture should provide a single access point for the Metadata repository. The access point must supply all related Metadata resources transparently to the user. Users should be able to access Metadata without being aware of the differing environments of the data sources. In analytics and Big Data solutions, the interface may have largely user-defined functions (UDF) to draw on various data sets, and the Metadata exposure to the end user is inherent to those customizations. With less reliance on UDF in solutions, end users will be gathering, inspecting, and using data sets more directly and various supporting Metadata is usually more exposed.

Design of the architecture depends on the specific requirements of the organization. Three technical architectural approaches to building a common Metadata repository mimic the approaches to designing data warehouses: centralized, distributed, and hybrid (see Section 1.3.6). These approaches all take into account implementation of the repository, and how the update mechanisms operate.

Create a data model for the Metadata repository, or metamodel, as one of the first design steps after the Metadata strategy is complete and the business requirements are understood. Different levels of metamodel may be developed as needed; a high-level conceptual model, that explains the relationships between systems, and a lower level metamodel that details the attributions, to describe the elements and processes of a model. In addition to being a planning tool and a means of articulating requirements, the metamodel is in itself a valuable source of Metadata. this figure depicts a sample Metadata repository metamodel. The boxes represent the high-level major entities, which contain the data.

DMBoK Figure 87 Hybrid Metadata Architecture

Wed, 04 Dec 2024 20:04:58 GMT

Another advanced architectural approach is bi-directional Metadata Architecture, which allows Metadata to change in any part of the architecture (source, data integration, user interface), and then feedback is coordinated from the repository (broker) into its original source.

Various challenges are apparent in this approach. The design forces the Metadata repository to contain the latest version of the Metadata source and forces it to manage changes to the source, as well. Changes must be trapped systematically, and then resolved. Additional sets of process interfaces to tie the repository back to the Metadata source(s) must be built and maintained.

This figure illustrates how common Metadata from different sources is collected in a centralized Metadata store. Users submit their queries to the Metadata portal, which passes the request to a centralized repository. The centralized repository will try to fulfill the user request from the common Metadata collected initially from the various sources. As the request becomes more specific or the user needs more detailed Metadata then the centralized repository will delegate down to the specific source to research the specific details. Global search across the various tools is available due to the common Metadata collected in the centralized repository.

DMBoK Figure 86 Distributed Metadata Architecture

Wed, 27 Nov 2024 14:00:00 GMT

A completely distributed architecture maintains a single access point. The Metadata retrieval engine responds to user requests by retrieving data from source systems in real time; there is no persistent repository. In this architecture, the Metadata management environment maintains the necessary source system catalogs and lookup information needed to process user queries and searches effectively. A common object request broker or similar middleware protocol accesses these source systems.

Advantages of distributed Metadata Architecture include:

Metadata is always as current and valid as possible because it is retrieved from its source
Queries are distributed, possibly improving response and process time
Metadata requests from proprietary systems are limited to query processing rather than requiring a detailed understanding of proprietary data structures, therefore minimizing the implementation and maintenance effort required
Development of automated Metadata query processing is likely simpler, requiring minimal manual intervention
Batch processing is reduced, with no Metadata replication or synchronization processes

Distributed architectures also have limitations:

No ability to support user-defined or manually inserted Metadata entries since there is no repository in which to place these additions
Standardization of presenting Metadata from various systems
Query capabilities are directly affected by the availability of the participating source systems
The quality of Metadata depends solely on the participating source systems

This figure illustrates a distributed Metadata Architecture. There is no centralized Metadata repository store and the portal passes the users’ requests to the appropriate tool to execute. As there is no centralized store for the Metadata to be collected from the various tools, every request has to be delegated down to the sources; hence, no capability exist for a global search across the various Metadata sources.

Welcome New Members!

Tue, 26 Nov 2024 14:00:00 GMT

A WARM DAMA Rocky Mountain Chapter welcome to our new members who joined in Q3! We wouldn't be the organization we are without you!

Professional Members:

Adefunke (Funke) B
Aileen P
Allen H
Cheryl B
Christopher H
John C
John L
Kris N
Nand S
Reeves S
Robert M
Terry T
Victoria B
Whitney C

Corporate Members:

Diana C with Cambium
Charles W with The Doyle Group

Guest Members:

Abdoulie C
Adrian C
Alex F
Allabaksh S
Alyssa P
Andrew T
Andrey S
Angelo S
Angelo V
Audrey S
Bikiran M
Bob M
Bryson T
Caleb S
Camiya I
DuWayne B
Ellie N
Esteban M
Ethan M
Gaius M
George M
Hannah C
Hasina R
Ian G
Jessica T
Jorge V
Kamal m
Karen M
Kathryn D
Katya K
Kevin J
Kristen K
Kristy T
Mantikoe M
Mary Kate B
Mary N
Meghan V
Mouyseang A
Mubeena K
Muhsin K
Natalie C
Neeraj Kumar J
Patrick V
Robert C
Robert L
Robert T
Rod G
Sagar B
Sheetal D
Susan H
Tarini S
Tessa H
Tessa K
Vanessa S
Wes B
Wes S
YuChen L
Yvette F
Zachery B

Thank you also to everyone who renewed their membership in Q3!

November 2024 Newsletter

Sat, 23 Nov 2024 02:00:00 GMT

November 2024 Newsletter.pdf

Welcome New Board Member - Mandi Albano

Thu, 21 Nov 2024 14:00:00 GMT

Mandi Albano joins the DAMA-RMC board as the new VP of Data.

Amanda (Mandi) Albano is a seasoned software and database expert with a passion for leveraging technology to drive business success and improve lives. With a foundation in complex system design and data management, she began her career at StarTek Inc., developing performance-enhancing software supporting major telecommunications companies. Amanda then transitioned to consulting at Sogeti USA, where she led projects for the State of Wyoming, focusing on data integration and reporting. For the past 14 years at Market Perceptions, Inc., she has specialized in creating data-driven solutions centered around strategic and operational insights based on marking research data. Amanda combines her technical expertise with a commitment to building strong, trust-based partnerships, aiming to deliver best-in-class solutions that foster customer growth and advancement.

Please give Mandi a warm DAMA-RMC welcome.

Mandi Albano Linked In

DMBok Figure 85 Centralized Metadata Architecture

Wed, 20 Nov 2024 14:00:00 GMT

A centralized architecture consists of a single Metadata repository that contains copies of Metadata from the various sources. Organizations with limited IT resources or those seeking to automate as much as possible, may choose to avoid this architecture option. Organizations seeking a high degree of consistency within the common Metadata repository can benefit from a centralized architecture.

Advantages of a centralized repository include:

High availability, since it is independent of the source systems
Quick Metadata retrieval, since the repository and the query reside together
Resolved database structures not affected by the proprietary nature of third party or commercial systems
Extracted Metadata may be transformed, customized, or enhanced with additional Metadata that may not reside in the source system, improving quality

Some limitations of the centralized approach include:

Complex processes are necessary to ensure that changes in source Metadata are quickly replicated into the repository
Maintenance of a centralized repository can be costly
Extraction could require custom modules or middleware
Validation and maintenance of customized code can increase the demands on both internal IT staff and the software vendors

This figure shows how Metadata is collected in a standalone Metadata repository with its own internal Metadata store. The internal store is populated through a scheduled import (arrows) of the Metadata from the various tools. In turn, the centralized repository exposes a portal for the end users to submit their queries. The Metadata portal passes the request to the centralized Metadata repository. The centralized repository will fulfill the request from the collected Metadata. In this type of implementation, the capability to pass the request from the user to various tools directly is not supported. Global search across the Metadata collected from the various tool is possible due to the collection of various Metadata in the centralized repository.

ALERT - DMBoK Scam

Fri, 15 Nov 2024 14:00:00 GMT

It has come to our attention that a nefarious scammer has created a fraudulent google site with a variation of our DAMA-RMC name and is attempting to scam people by pretending to sell #DMBoKs.

Please note that the only valid site to purchase DMBoKs through DAMA Rocky Mountain Chapter is via our direct website: https://damarmc.org/ through the DMBoK Order page: https://damarmc.org/dmbok, which can only be accessed after you have logged in with your professional membership login and password.

We do not use square.site or squareup.com as our payment processing service.

We are attempting to get the site shut down, however that may take time. If you have ordered a DMBoK through any site other than DAMA International, Amazon, or the one above, please report the charge to your credit card company or bank as fraudulent.

Please reach out with any questions or concerns to Communications@damarmc.org.

We hope this does not impact any members in the future. Thank you for your attention to this matter!

DMBoK Figure 84 Context Diagram: Metadata

Wed, 13 Nov 2024 18:03:54 GMT

The most common definition of Metadata, “data about data,” is misleadingly simple. The kind of information that can be classified as Metadata is wide-ranging. Metadata includes information about technical and business processes, data rules and constraints, and logical and physical data structures. It describes the data itself (e.g., databases, data elements, data models), the concepts the data represents (e.g., business processes, application systems, software code, technology infrastructure), and the connections (relationships) between the data and concepts. Metadata helps an organization understand its data, its systems, and its workflows. It enables Data Quality assessment and is integral to the management of databases and other applications. It contributes to the ability to process, maintain, integrate, secure, audit, and govern other data.

To understand Metadata’s vital role in data management, imagine a large library, with hundreds of thousands of books and magazines, but no card catalog. Without a card catalog, readers might not even know how to start looking for a specific book or even a specific topic. The card catalog not only provides the necessary information (which books and materials the library owns and where they are shelved) it also enables patrons to find materials using different starting points (subject area, author, or title). Without the catalog, finding a specific book would be difficult if not impossible. An organization without Metadata is like a library without a card catalog.

Metadata is essential to data management as well as data usage (see multiple references to Metadata throughout the DAMA-DMBOK). All large organizations produce and use a lot of data. Across an organization, different individuals will have different levels of data knowledge, but no individual will know everything about the data. This information must be documented or the organization risks losing valuable knowledge about itself. Metadata provides the primary means of capturing and managing organizational knowledge about data. However, Metadata management is not only a knowledge management challenge; it is also a risk management necessity. Metadata is necessary to ensure an organization can identify private or sensitive data and that it can manage the data lifecycle for its own benefit and in order to meet compliance requirements and minimize risk exposure.

Without reliable Metadata, an organization does not know what data it has, what the data represents, where it originates, how it moves through systems, who has access to it, or what it means for the data to be of high quality. Without Metadata, an organization cannot manage its data as an asset. Indeed, without Metadata, an organization may not be able to manage its data at all. As technology has evolved, the speed at which data is generated has also increased. Technical Metadata has become integral to the way in which data is moved and integrated. ISO’s Metadata Registry Standard, ISO/IEC 11179, is intended to enable Metadata-driven exchange of data in a heterogeneous environment, based on exact definitions of data. Metadata present in XML and other formats enables use of the data. Other types of Metadata tagging allow data to be exchanged while retaining signifiers of ownership, security requirements, etc. (See Chapter 8.)

Like other data, Metadata requires management. As the capacity of organizations to collect and store data increases, the role of Metadata in data management grows in importance. To be data-driven, an organization must be Metadata-driven.

DMBoK Figure 83 Release Process Example

Wed, 06 Nov 2024 14:00:00 GMT

Release Management is critical to an incremental development processes that grows new capabilities, enhances the production deployment, and ensures provision of regular maintenance across the deployed assets. This process will keep the warehouse up-to-date, clean, and operating at its best. However, this process requires the same alignment between IT and Business as between the Data Warehouse model and the BI capabilities. It is a continual improvement effort.

This Figure illustrates an example release process, based on a quarterly schedule. Over the year, there are three business-driven releases and one technology-based release (to address requirements internal to the warehouse). The process should enable incremental development of the warehouse and management of the backlog of requirements.

DMBoK Figure 82 Conceptual DW/BI and Big Data Architecture

Wed, 30 Oct 2024 13:00:00 GMT

The data warehouse environment includes a collection of architectural components that need to be organized to meet the needs of the enterprise. Figure 82 depicts the architectural components of the DW/BI and Big Data Environment discussed in this section. The evolution of Big Data has changed the DW/BI landscape by adding another path through which data may be brought into an enterprise.

This Figure also depicts aspects of the data lifecycle. Data moves from source systems into a staging area where it may be cleansed and enriched as it is integrated and stored in the DW and/or an ODS. From the DW, it may be accessed via marts or cubes and used for various kinds of reporting. Big Data goes through a similar process but with a significant difference: while most warehouses integrate data before landing it in tables, Big Data solutions ingest data before integrating it. Big Data BI may include predictive analytics and data mining, as well as more traditional forms of reporting. (See Chapter 14.)

Source Systems, on the left side of this Figure, include the operational systems and external data to be brought into the DW/BI environment. These typically include operational systems such as CRM, Accounting, and Human Resources applications, as well as operational systems that differ based on industry. Data from vendors and external sources may also be included, as may DaaS, web content, and any Big Data computation results.

Data integration covers Extract, Transform, and Load (ETL), data virtualization, and other techniques of getting data into a common form and location. In a SOA environment, the data services layers are part of this component. In this Figure, all the arrows represent data integration processes. (See Chapter 8.)

DMBoK Figure 81 Kimball's Data Warehouse Chess Pieces

Wed, 23 Oct 2024 13:00:00 GMT

Kimball’s Dimensional Data Warehouse is the other primary pattern for DW development. Kimball defines a data warehouse simply as “a copy of transaction data specifically structured for query and analysis” (Kimball, 2002). The ‘copy’ is not exact, however. Warehouse data is stored in a dimensional data model. The dimensional model is designed to enable data consumers to understand and use the data, while also enabling query performance. It is not normalized in the way an entity relationship model is.

Often referred to as Star Schema, dimensional models are comprised of facts, which contain quantitative data about business processes (e.g., sales numbers), and dimensions, which store descriptive attributes related to fact data and allow data consumers to answer questions about the facts (e.g., how many units of product X were sold this quarter?) A fact table joins with many dimension tables, and when viewed as a diagram, appears as a star. (See Chapter 5.) Multiple fact tables will share the common, or conformed, dimensions via a ‘bus’, similar to a bus in a computer. Multiple data marts can be integrated at an enterprise level by plugging into the bus of conformed dimensions.

The DW bus matrix shows the intersection of business processes that generate fact data and data subject areas that represent dimensions. Opportunities for conformed dimensions exist where multiple processes use the same data. Table 27 is a sample bus matrix. In this example, the business processes for Sales, Inventory, and Orders all require Date and Product data. Sales and Inventory both require Store data, while Inventory and Orders require Vendor data. Date, Product, Store, and Vendor are all candidates for conformed dimensions. In contrast, Warehouse is not shared; it is used only by Inventory.

The enterprise DW bus matrix can be used to represent the long-term data content requirements for the DW/BI system, independent of technology. This tool enables an organization to scope manageable development efforts. Each implementation builds an increment of the overall architecture. At some point, enough dimensional schemas exist to make good on the promise of an integrated enterprise data warehouse environment. This figure represents Kimball’s Data Warehouse Chess Pieces view of DW/BI architecture. Note that Kimball’s Data Warehouse is more expansive than Inmon’s. The DW encompasses all components in the data staging and data presentation areas.

Operational source systems: Operational / transactional applications of the Enterprise. These create the data that is integrated into the ODS and DW. This component is equivalent to the application systems in the CIF diagram.

Data staging area: Kimball’s staging includes the set of processes needed to integrate and transform data for presentation. It can be compared to a combination of CIF’s integration, transformation, and DW components. Kimball’s focus is on efficient end-delivery of the analytical data, a scope smaller than Inmon’s corporate management of data. Kimball’s enterprise DW can fit into the architecture of the data staging area.

Data presentation area: Similar to the Data Marts in the CIF. The key architectural difference being an integrating paradigm of a ‘DW Bus,’ such as shared or conformed dimensions unifying the multiple data marts.

Data access tools: Kimball’s approach focuses on end users’ data requirements. These needs drive the adoption of appropriate data access tools.

October 2024 Newsletter

Fri, 18 Oct 2024 13:00:00 GMT

October 2024 Newsletter.pdf

DMBoK Figure 80 The Corporate Information Factory

Wed, 16 Oct 2024 13:00:00 GMT

Bill Inmon’s Corporate Information Factory (CIF) is one of the two primary patterns for data warehousing. The component parts of Inmon’s definition of a data warehouse, “a subject oriented, integrated, time variant, and nonvolatile collection of summary and detailed historical data,” describe the concepts that support the CIF and point to the differences between warehouses and operational systems.

Subject-oriented: The data warehouse is organized based on major business entities, rather than focusing on a functional or application.

Integrated: Data in the warehouse is unified and cohesive. The same key structures, encoding and decoding of structures, data definitions, naming conventions are applied consistently throughout the warehouse. Because data is integrated, Warehouse data is not simply a copy of operational data. Instead, the warehouse becomes a system of record for the data.

Time variant: The data warehouse stores data as it exists in a set point in time. Records in the DW are like snapshots. Each one reflects the state of the data at a moment of time. This means that querying data based on a specific time period will always produce the same result, regardless of when the query is submitted.

Non-volatile: In the DW, records are not normally updated as they are in operational systems. Instead, new data is appended to existing data. A set of records may represent different states of the same transaction.

Aggregate and detail data: The data in the DW includes details of atomic level transactions, as well as summarized data. Operational systems rarely aggregate data. When warehouses were first established, cost and space considerations drove the need to summarize data. Summarized data can be persistent (stored in a table) or non-persistent (rendered in a view) in contemporary DW environments. The deciding factor in whether to persist data is usually performance.

Historical: The focus of operational systems is current data. Warehouses contain historical data as well. Often they house vast amounts of it.

Inmon, Claudia Imhoff and Ryan Sousa describe data warehousing in the context of the Corporate Information Factory (CIF). See this figure. CIF components include:

Applications: Applications perform operational processes. Detail data from applications is brought into the data warehouse and the operational data stores (ODS) where it can be analyzed.

Staging Area: A database that stands between the operational source databases and the target databases. The data staging area is where the extract, transform, and load effort takes place. It is not used by end users. Most data in the data staging area is transient, although typically there is some relatively small amount of persistent data.

Integration and transformation: In the integration layer, data from disparate sources is transformed so that it can be integrated into the standard corporate representation / model in the DW and ODS.

Operational Data Store (ODS): An ODS is integrated database of operational data. It may be sourced directly from applications or from other databases. ODS’s generally contain current or near term data (30-90 days), while a DW contains historical data as well (often several years of data). Data in ODS’s is volatile, while warehouse data is stable. Not all organizations use ODS’s. They evolved as to meet the need for low latency data. An ODS may serve as the primary source for a data warehouse; it may also be used to audit a data warehouse.

Data marts: Data marts provide data prepared for analysis. This data is often a sub-set of warehouse data designed to support particular kinds of analysis or a specific group of data consumers. For example, marts can aggregate data to support faster analysis. Dimensional modeling (using denormalization techniques) is often used to design user-oriented data marts.

Operational Data Mart (OpDM): An OpDM is a data mart focused on tactical decision support. It is sourced directly from an ODS, rather than from a DW. It shares characteristics of the ODS: it contains current or near-term data. Its contents are volatile.

Data Warehouse: The DW provides a single integration point for corporate data to support management decision-making, and strategic analysis and planning. The data flows into a DW from the application systems and ODS, and flows out to the data marts, usually in one direction only. Data that needs correction is rejected, corrected at its source, and ideally re-fed through the system.

Operational reports: Reports are output from the data stores.

Reference, Master, and external data: In addition to transactional data from applications, the CIF also includes data required to understand transactions, such as reference and Master Data. Access to common data simplifies integration in the DW. While applications consume current master and Reference Data, the DW also requires historical values and the timeframes during which they were valid (see Chapter 10).

This figure depicts movement within the CIF, from data collection and creation via applications (on the left) to the creation of information via marts and analysis (on the right). Movement from left to right includes other changes. For example,

The purpose shifts from execution of operational functions to analysis
End users of systems move from front line workers to decision-makers
System usage moves from fixed operations to ad hoc uses
Response time requirements are relaxed (strategic decisions take more time than do daily operations)
Much more data is involved in each operation, query, or process

The data in DW and marts differs from that in applications:

Data is organized by subject rather than function
Data is integrated data rather than ‘siloed’
Data is time-variant vs. current-valued only
Data has higher latency in DW than in applications
Significantly more historical data is available in DW than in applications

DMBoK Figure 79 Context Diagram DW/BI

Wed, 09 Oct 2024 13:30:00 GMT

The concept of the Data Warehouse emerged in the 1980s as technology enabled organizations to integrate data from a range of sources into a common data model. Integrated data promised to provide insight into operational processes and open up new possibilities for leveraging data to make decisions and create organizational value. As importantly, data warehouses were seen as a means to reduce the proliferation of decision support systems (DSS), most of which drew on the same core enterprise data. The concept of an enterprise warehouse promised a way to reduce data redundancy, improve the consistency of information, and enable an enterprise to use its data to make better decisions.

Data warehouses began to be built in earnest in the 1990s. Since then (and especially with the co-evolution of Business Intelligence as a primary driver of business decision-making), data warehouses have become ‘mainstream’. Most enterprises have data warehouses and warehousing is the recognized core of enterprise data management.63 Even though well established, the data warehouse continues to evolve. As new forms of data are created with increasing velocity, new concepts, such as data lakes, are emerging that will influence the future of the data warehouse. See Chapters 8 and 15.

The primary driver for data warehousing is to support operational functions, compliance requirements, and Business Intelligence (BI) activities (though not all BI activities depend on warehouse data). Increasingly organizations are being asked to provide data as evidence that they have complied with regulatory requirements. Because they contain historical data, warehouses are often the means to respond to such requests. Nevertheless, Business Intelligence support continues to be the primary reason for a warehouse. BI promises insight about the organization, its customers, and its products. An organization that acts on knowledge gained from BI can improve operational efficiency and competitive advantage. As more data has become available at a greater velocity, BI has evolved from retrospective assessment to predictive analytics.

DMBoK Figure 78 Reference Data Change Request Process

Wed, 02 Oct 2024 13:30:00 GMT

Since Reference Data is a shared resource, it cannot be changed arbitrarily. The key to successful Reference Data Management is organizational willingness to relinquish local control of shared data. To sustain this support, provide channels to receive and respond to requests for changes to Reference Data. The Data Governance Council should ensure that policies and procedures are implemented to handle changes to data within reference and Master Data environments.

Changes to Reference Data will need to be managed. Minor changes may affect a few rows of data. For example, when the Soviet Union broke into independent states, the term Soviet Union was deprecated and new codes were added. In the healthcare industry, procedure and diagnosis codes are updated annually to account for refinement of existing codes, obsoleting of codes, and the introduction of new codes. Major revisions to Reference Data impact data structure. For example, ICD-10 Diagnostic Codes are structured in ways very different from ICD-9. ICD10 has a different format. There are different values for the same concepts. More importantly, ICD-10 has additional principles of organization. ICD10 codes have a different granularity and are much more specific, so more information is conveyed in a single code. Consequently, there are many more of them (as of 2015, there were 68,000 ICD-10 codes, compared with 13,000 ICD-9s).

The mandated use of ICD-10 codes in the US in 2015 required significant planning. Healthcare companies needed to make system changes as well as adjustments to impacted reporting to account for the new standard.

Types of changes include:

Row level changes to external Reference Data sets
Structural changes to external Reference Data sets
Row level changes to internal Reference Data sets
Structural changes to internal Reference Data sets
Creation of new Reference Data sets

Changes can be planned / scheduled or ad hoc. Planned changes, such as monthly or annual updates to industry standard codes, require less governance than ad hoc updates. The process to request new Reference Data sets should account for potential uses beyond those of the original requestor.

Change requests should follow a defined process, as illustrated in this figure. When requests are received, stakeholders should be notified so that impacts can be assessed. If changes need approval, discussions should be held to get that approval. Changes should be communicated.

DMBoK Figure 77 Master Data Sharing Architecture Example

Wed, 25 Sep 2024 16:32:06 GMT

There are several basic architectural approaches to reference and Master Data integration. Each Master Data subject area will likely have its own system of record. For example, the human resource system usually serves as the system of record for employee data. A CRM system might serve as the system of record for customer data, while an ERP system might serve as the system of record for financial and product data.

The data sharing hub architecture model shown in this figure represents a hub-and-spoke architecture for Master Data. The Master Data hub can handle interactions with spoke items such as source systems, business applications, and data stores while minimizing the number of integration points. A local data hub can extend and scale the Master Data hub. (See Chapter 8.)

Each of the three basic approaches to implementing a Master Data hub environment has pros and cons:

A Registry is an index that points to Master Data in the various systems of record. The systems of record manage Master Data local to their applications. Access to Master Data comes from the master index. A registry is relatively easy to implement because it requires few changes in the systems of record. But often, complex queries are required to assemble Master Data from multiple systems. Moreover, multiple business rules need to be implemented to address semantic differences across systems in multiple places.

In a Transaction Hub, applications interface with the hub to access and update Master Data. The Master Data exists within the Transaction Hub and not within any other applications. The Transaction Hub is the system of record for Master Data. Transaction Hubs enable better governance and provide a consistent source of Master Data. However, it is costly to remove the functionality to update Master Data from existing systems of record. Business rules are implemented in a single system: the Hub.

A Consolidated approach is a hybrid of Registry and Transaction Hub. The systems of record manage Master Data local to their applications. Master Data is consolidated within a common repository and made available from a data-sharing hub, the system of reference for Master Data. This eliminates the need to access directly from the systems of record. The Consolidated approach provides an enterprise view with limited impact on systems of record. However, it entails replication of data and there will be latency between the hub and the systems of record.

September 2024 Newsletter

Fri, 20 Sep 2024 13:00:00 GMT

September 2024 Newsletter.pdf

DMBoK Figure 76 Key Processing Steps for MDM

Wed, 18 Sep 2024 13:00:00 GMT

Key processing steps for MDM are illustrated in this Figure. They include data model management; data acquisition; data validation, standardization, and enrichment; entity resolution; and stewardship and sharing.

In a comprehensive MDM environment, the logical data model will be physically instantiated in multiple platforms. It guides the implementation of the MDM solution, providing the basis of data integration services. It should guide how applications are configured to take advantage of data reconciliation and data quality verification capabilities.

DMBoK Figure 75 Context Diagram: Reference and Master Data

Wed, 11 Sep 2024 13:00:00 GMT

In any organization, certain data is required across business areas, processes, and systems. The overall organization and its customers benefit if this data is shared and all business units can access the same customer lists, geographic location codes, business unit lists, delivery options, part lists, accounting cost center codes, governmental tax codes, and other data used to run the business. People using data generally assume a level of consistency exists across the organization, until they see disparate data.

In most organizations, systems and data evolve more organically than data management professionals would like. Particularly in large organizations, various projects and initiatives, mergers and acquisitions, and other business activities result in multiple systems executing essentially the same functions, isolated from each other. These conditions inevitably lead to inconsistencies in data structure and data values between systems. This variability increases costs and risks. Both can be reduced through the management of Master Data and Reference Data.

DMBoK Figure 74 Information Governance Reference Model

Wed, 04 Sep 2024 13:00:00 GMT

Documents, records, and other unstructured content represent risk to an organization. Managing this risk and getting value from this information both require governance. Drivers include:

Legal and regulatory compliance
Defensible disposition of records
Proactive preparation for e-discovery
Security of sensitive information
Management of risk areas such as email and Big Data

Principles of successful Information Governance programs are emerging. One set of principles is the ARMA GARP® principles (see Section 1.2). Other principles include:

Assign executive sponsorship for accountability
Educate employees on information governance responsibilities
Classify information under the correct record code or taxonomy category
Ensure authenticity and integrity of information
Determine that the official record is electronic unless specified differently
Develop policies for alignment of business systems and third-parties to information governance standards
Store, manage, make accessible, monitor, and audit approved enterprise repositories and systems for records and content
Secure confidential or personally identifiable information
Control unnecessary growth of information
Dispose information when it reaches the end of its lifecycle
Comply with requests for information (e.g., discovery, subpoena, etc.)
Improve continuously

The Information Governance Reference Model (IGRM) shows the relationship of Information Governance to other organizational functions. The outer ring includes the stakeholders who put policies, standards, processes, tools and infrastructure in place to manage information. The center shows a lifecycle diagram with each lifecycle component within the color or colors of the stakeholder(s) who executes that component. The IGRM complements ARMA’s GARP®.

Sponsorship by someone close to or within the ‘C’ suite is a critical requirement for the formation and sustainability of the Information Governance program. A cross-functional senior level Information Council or Steering Committee is established that meets on a regular basis. The Council is responsible for an enterprise Information Governance strategy, operating procedures, guidance on technology and standards, communications and training, monitoring, and funding. Information Governance policies are written for the stakeholder areas, and then ideally technology is applied for enforcement.

DMBoK Figure 73 Electronic Discovery Reference Model

Wed, 28 Aug 2024 13:00:00 GMT

Discovery is a legal term that refers to pre-trial phase of a lawsuit where both parties request information from each other to find facts for the case and to see how strong the arguments are on either side. The US Federal Rules of Civil Procedure (FRCP) have governed the discovery of evidence in lawsuits and other civil cases since 1938. For decades, paper-based discovery rules were applied to e-discovery. In 2006, amendments to the FRCP accommodated the discovery practice and requirements of ESI in the litigation process.

Other global regulations have requirements specific to the ability of an organization to produce electronic evidence. Examples include the UK Bribery Act, Dodd-Frank Act, Foreign Account Tax Compliance Act (FATCA), Foreign Corrupt Practices Act, EU Data Protection Regulations and Rules, global anti-trust regulations, sector-specific regulations, and local court procedural rules.

Electronic documents usually have Metadata (which may not be available for paper documents) that plays an important part in evidence. Legal requirements come from the key legal processes such as e-discovery, as well as data and records retention practices, the legal hold notification (LHN) process, and legally defensible disposition practices. LHN includes identifying information that may be requested in a legal proceeding, locking that data or document down to prevent editing or deletion, and then notifying all parties in an organization that the data or document in question is subject to a legal hold.

This figure depicts a high-level Electronic Discovery Reference Model developed by EDRM, a standards and guidelines organization for e-discovery. This framework provides an approach to e-discovery that is handy for people involved in identifying how and where the relevant internal data is stored, what retention policies apply, what data is not accessible, and what tools are available to assist in the identification process.

The EDRM model assumes that data or information governance is in place. The model includes eight e-discovery phases that can be iterative. As e-discovery progresses, the volume of discoverable data and information is greatly reduced as their relevance is greatly increased.

The first phase, Identification, has two sub-phases: Early Case Assessment and Early Data Assessment (not depicted in the diagram). In Early Case Assessment, the legal case itself is assessed for pertinent information, called descriptive information or Metadata (e.g., keywords, date ranges, etc.). In Early Data Assessment, the types and location of data relevant to the case is assessed. Data assessment should identify policies related to the retention or destruction of relevant data so that ESI can be preserved. Interviews should be held with records management personnel, data custodians or data owners, and information technology personnel to obtain pertinent information. In addition, the involved personnel need to understand the case background, legal hold, and their role in the litigation.

The next phases in the model are the Preservation and Collection. Preservation ensures that the data that has been identified as potentially relevant is placed in a legal hold so it is not destroyed. Collection includes the acquisition and transfer of identified data from the company to their legal counsel in a legally defensible manner.

During the Processing phase data is de-duplicated, searched, and analyzed to determine which data items will move forward to the Review phase. In the Review phase, documents are identified to be presented in response to the request. Review also identifies privileged documents that will be withheld. Much of the selection depends on Metadata associated with the documents. Processing takes place after the Review phase because it addresses content analysis to understand the circumstances, facts and potential evidence in litigation or investigation and to enhance the search and review processes.

Processing and Review depend on analysis, but Analysis is called out as a separate phase with a focus on content. The goal of content analysis is to understand the circumstances, facts, and potential evidence in litigation or investigation, in order to formulate a strategy in response to the legal situation.

In the Production phase, data and information are turned over to opposing counsel, based on agreed-to specifications. Original sources of information may be files, spreadsheets, email, databases, drawings, photographs, data from proprietary applications, website data, voicemail, and much more. The ESI can be collected, processed and output to a variety of formats. Native production retains the original format of the files. Near-native production alters the original format through extraction and conversion. ESI can be produced in an image, or near paper, format. Fielded data is Metadata and other information extracted from native files when ESI is processed and produced in a text-delimited file or XML load file. The lineage of the materials provided during the Production phase is important, because no one wants to be accused of altering data or information provided.

Displaying the ESI at depositions, hearings, and trials is part of the Presentation phase. The ESI exhibits can be presented in paper, near paper, near-native and native formats to support or refute elements of the case. They may be used to elicit further information, validate existing facts or positions, or persuade an audience.

August 2024 Newsletter

Fri, 23 Aug 2024 13:00:00 GMT

August 2024 Newsletter.pdf

DMBoK Figure 72 Document Hierarchy based on ISO 9001-4.2

Wed, 21 Aug 2024 13:00:00 GMT

Document management includes records management. Managing records has special requirements. Records management includes the full lifecycle: from record creation or receipt through processing, distribution, organization, and retrieval, to disposition. Records can be physical (e.g., documents, memos, contracts, reports or microfiche); electronic (e.g., email content, attachments, and instant messaging); content on a website; documents on all types of media and hardware; and data captured in databases of all kinds. Hybrid records, such as aperture cards (paper record with a microfiche window imbedded with details or supporting material), combine formats. A Vital Record is type a record required to resume an organization’s operations the event of a disaster.

Trustworthy records are important not only for record keeping but also for regulatory compliance. Having signatures on the record contributes to a record’s integrity. Other integrity actions include verification of the event (i.e., witnessing in real time) and double-checking the information after the event.

Well-prepared records have characteristics such as:

Content: Content must be accurate, complete and truthful.
Context: Descriptive information (Metadata) about the record’s creator, date of creation, or relationship to other records should be collected, structured and maintained with the record at the time of record creation.
Timeliness: A record should be created promptly after the event, action or decision occurs.
Permanency: Once they are designated as records, records cannot be changed for the legal length of their existence.
Structure: The appearance and arrangement of a record’s content should be clear. They should be recorded on the correct forms or templates. Content should be legible, terminology should be used consistently.

Many records exist in both electronic and paper formats. Records Management requires the organization to know which copy (electronic or paper) is the official ‘copy of record’ to meet record keeping obligations. Once the copy of record is determined, the other copy can be safely destroyed.

Upcoming Changes to CDMP Certification and DMBOK

Wed, 14 Aug 2024 15:31:31 GMT

Exciting updates are coming to the Certified Data Management Professional (CDMP) certification exams starting in October 2024. The exams will now cover the DAMA-DMBOK2 Revised Edition, an updated and improved version of the Data Management Body of Knowledge. This revised edition addresses known inconsistencies and inaccuracies, making it a more reliable and comprehensive resource for data management professionals. For those preparing for the CDMP certification, it's important to note that this revised edition will be the new reference material.

The DAMA-DMBOK2 Revised Edition has been created to ensure the content remains relevant and accurate for data management practitioners. DAMA International embarked on this update to improve upon the previous version, incorporating feedback from members and volunteers. The revised edition aims to provide a more consistent and precise framework, making it easier for professionals to understand and apply the principles of data management.

Key improvements in the DAMA-DMBOK2 Revised Edition include standardized terminology and acronyms, corrections of typos and errors, improvements to context diagrams, and enhancements to the Data Quality chapter. These updates ensure the content is not only accurate but also easier to comprehend and apply. The revised edition is available for purchase (and discounted for Professional Members) through DAMA-RMC at DMBoK.

For those currently preparing for the CDMP certification, you can continue using the DAMA-DMBOK 2nd Edition until October 2024. After that date, the revised edition will be the authoritative resource.

We encourage you to join one of our upcoming 30-minute informational sessions on the CDMP Study Group during the week of August 26th. You can also enroll in our 12-week virtual study program which will start in September. To enroll in the study program, you must be a Professional Member anyone can attend the informational sessions. We also have a self-paced study option available.

Information session links:

Tuesday, August 27 at 6pm MST
- DAMA-RMC CDMP Information Session 1 - ZOOM LINK
- DAMA-RMC CDMP Information Session 1 - ADD TO CALENDAR

Wednesday, August 28 at 12:30 MST
- DAMA-RMC CDMP Information Session 2 - ZOOM LINK
- DAMA-RMC CDMP Information Session 2 - ADD TO CALENDAR

For more information, please contact jhorner@dama-rockymountainchapter.org or visit the CDMP Webpage. Additionally, please update your DAMA RMC profile to indicate your interest in the CDMP so we can keep you update to date on all things DMBoK and CDMP.

Join Us for a 30-Minute CDMP Informational Session

Wed, 14 Aug 2024 15:22:53 GMT

We are pleased to invite you to a concise and informative session on the Certified Data Management Professional (CDMP) certification. This 30-minute meeting will cover essential topics and recent updates, ensuring you are well-prepared for the certification process.

The study sessions will run from the week of September 9th - December (Meeting time and day of week TBD tentatively planning for Wednesday at 6pm)

At the conclusion we will host to ½ day virtual prep reviews and host the annual pay if you pass event which allows attendees to only pay the exam fee if they pass.

**Agenda:**

1. **CDMP Exam Overview**

Description of the CDMP certification levels and exam structure.
Key topics and study resources.

2. **Study Session Logistics**

Registration, scheduling, and common logistical questions
Format

3. **New Changes to DMBOK / CDMP Exams**

Overview of recent updates to the Data Management Body of Knowledge (DMBOK) and changes in CDMP exam content.

4. **General Q&A**

**Date and Time:**

We are offering two 30 minute information sessions:

Tuesday, August 27 at 6pm MST
- DAMA-RMC CDMP Information Session 1 - ZOOM LINK
- DAMA-RMC CDMP Information Session 1 - ADD TO CALENDAR

Wednesday, August 28 at 12:30 MST
- DAMA-RMC CDMP Information Session 2 - ZOOM LINK
- DAMA-RMC CDMP Information Session 2 - ADD TO CALENDAR

Please RSVP to ProfessionalDevelopmentVP@damarmc.org as soon as possible to confirm your attendance. We look forward to your participation and engaging discussions.

DMBoK Figure 71 Context Diagram: Documents and Content

Sun, 11 Aug 2024 17:25:53 GMT

Document and Content Management entails controlling the capture, storage, access, and use of data and information stored outside relational databases. Its focus is on maintaining the integrity of and enabling access to documents and other unstructured or semi-structured information which makes it roughly equivalent to data operations management for relational databases. However, it also has strategic drivers. In many organizations, unstructured data has a direct relationship to structured data. Management decisions about such content should be applied consistently. In addition, as are other types of data, documents and unstructured content are expected to be secure and of high quality. Ensuring security and quality requires governance, reliable architecture, and well-managed Metadata.

The primary business drivers for document and content management include regulatory compliance, the ability to respond to litigation and e-discovery requests, and business continuity requirements. Good records management can also help organizations become more efficient. Well-organized, searchable websites that result from effective management of ontologies and other structures that facilitate searching help improve customer and employee satisfaction.

Laws and regulations require that organizations maintain records of certain kinds of activities. Most organizations also have policies, standards, and best practices for record keeping. Records include both paper documents and electronically stored information (ESI). Good records management is necessary for business continuity. It also enables an organization to respond in the case of litigation.

E-discovery is the process of finding electronic records that might serve as evidence in a legal action. As the technology for creating, storing, and using data has developed, the volume of ESI has increased exponentially. Some of this data will undoubtedly end up in litigation or regulatory requests.

The ability of an organization to respond to an e-discovery request depends on how proactively it has managed records such as email, chats, websites, and electronic documents, as well as raw application data and Metadata. Big Data has become a driver for more efficient e-discovery, records retention, and strong information governance.

Gaining efficiencies is a driver for improving document management. Technological advances in document management are helping organizations streamline processes, manage workflow, eliminate repetitive manual tasks, and enable collaboration. These technologies have the additional benefits of enabling people to locate, access, and share documents more quickly. They can also prevent documents from being lost. This is very important for e-discovery. Money is also saved by freeing up file cabinet space and reducing document handling costs.

DMBoK Figure 70 Enterprise Service Bus

Mon, 05 Aug 2024 20:06:26 GMT

An Enterprise Service Bus (ESB) is a system that acts as an intermediary between systems, passing messages between them. Applications can send and receive messages or files using the ESB, and are encapsulated from other processes existing on the ESB. An example of loose coupling, the ESB acts as the service between the applications.

DMBoK Figure 69 Application Coupling

Wed, 31 Jul 2024 13:00:00 GMT

Coupling describes the degree to which two systems are entwined. Two systems that are tightly coupled usually have a synchronous interface, where one system waits for a response from the other. Tight coupling represents a riskier operation: if one system is unavailable then they are both effectively unavailable, and the business continuity plan for both have to be the same.

Where possible, loose coupling is a preferred interface design, where data is passed between systems without waiting for a response and one system may be unavailable without causing the other to be unavailable. Loose coupling can be implemented using various techniques with services, APIs, or message queues. This figure illustrates a possible loose coupling design.

Service Oriented Architecture using an Enterprise Service Bus is an example of a loosely coupled data interaction design pattern.

Where the systems are loosely coupled, replacement of systems in the application inventory can theoretically be performed without rewriting the systems with which they interact, because the interaction points are well-defined.

DMBoK Figure 67 ETL Process Flow and Figure 68 ELT Process Flow

Wed, 24 Jul 2024 13:00:00 GMT

The load step of ETL is physically storing or presenting the result of the transformations in the target system.

Depending on the transformations performed, the target system’s purpose, and the intended use, the data may need further processing to be integrated with other data, or it may be in a final form, ready to present to consumers.

If the target system has more transformation capability than either the source or an intermediary application system, the order of processes may be switched to ELT – Extract, Load, and Transform. ELT allows transformations to occur after the load to the target system, often as part of the process. ELT allows source data to be instantiated on the target system as raw data, which can be useful for other processes. This is common in Big Data environments where ELT loads the data lake.

July 2024 Newsletter

Fri, 19 Jul 2024 13:00:00 GMT

July 2024 Newsletter.pdf

DMBoK Figure 66 Context Diagram: Data Integration and Interoperability

Wed, 17 Jul 2024 13:00:00 GMT

Data Integration and Interoperability (DII) describes processes related to the movement and consolidation of data within and between data stores, applications and organizations. Integration consolidates data into consistent forms, either physical or virtual. Data Interoperability is the ability for multiple systems to communicate. DII solutions enable basic data management functions on which most organizations depend:

Data migration and conversion
Data consolidation into hub or marts
Integration of vendor packages into an organization's application portfolio
Data sharing between applications and across organizations
Distributing data across data stores and data centers
Archiving data
Managing data interfaces
Obtaining and ingesting external data
Integrating structured and unstructured data
Providing operational intelligence and management decision support

DII is dependent on these other areas of data management:

Data Governance: For governing the transformation rules and message structures
Data Architecture: For designing solutions
Data Security: For ensuring solutions appropriately protect the security of data, whether it is persistent, virtual, or in motion between applications and organizations
Metadata: For tracking the technical inventory of data (persistent, virtual, and in motion), the business rules for transforming the data, and the operational history and lineage of the data
Data Storage and Operations: For managing the physical instantiation of the solutions
Data Modeling and Design: For designing the data structures including physical persistence in databases, virtual data structures, and messages passing information between applications and organizations

Data Integration and Interoperability is critical to Data Warehousing and Business Intelligence, as well as Reference Data and Master Data Management, because all of these focus on transforming and integrating data from source systems to consolidated data hubs and from hubs to the target systems where it can be delivered to data consumers, both system and human.

Data Integration and Interoperability is central to the emerging area of Big Data management. Big Data seeks to integrate various types of data, including data structured and stored in databases, unstructured text data in documents or files, other types of unstructured data such as audio, video, and streaming data. This integrated data can be mined, used to develop predictive models, and deployed in operational intelligence activities.

DMBoK Figure 65 Security Role Hierarchy Example Diagram

Wed, 10 Jul 2024 13:00:00 GMT

Security risks include elements that can compromise a network and/or database. The first step in identifying risk is identifying where sensitive data is stored, and what protections are required for that data. Evaluate each system for the following:

The sensitivity of the data stored or in transit
The requirements to protect that data, and
The current security protections in place

Document the findings, as they create a baseline for future evaluations. This documentation may also be a requirement for privacy compliance, such as in the European Union. Gaps must be remediated through improved security processes supported by technology. The impact of improvements should be measured and monitored to ensure risks are mitigated.

In larger organizations, white-hat hackers may be hired to assess vulnerabilities. A white hat exercise can be used as proof of an organization’s impenetrability, which can be used in publicity for market reputation.

DMBoK Figure 64 DMZ Example

Wed, 03 Jul 2024 13:00:00 GMT

Short for de-militarized zone, a is an area on the edge or perimeter of an organization, with a firewall between it and the organization. A DMZ environment will always have a firewall between it and the internet (see this figure). DMZ environments are used to pass or temporarily store data moving between organizations.

Please Welcome Our July Corporate Meeting Host Sponsor - The Doyle Group

Tue, 02 Jul 2024 13:00:00 GMT

The Doyle Group is an IT Consulting and Placement firm known for their strategic talent solutions and consultative approach.

With deep roots in the Denver community, the Doyle Group serves clients locally and across the United States. Bringing over 50 years of collective industry experience, they have built a reputation for delivering solutions tailored to meet each client's unique needs. Their mission is to forge meaningful partnerships with clients seeking top technology talent and to support highly skilled candidates in finding their next career opportunity. By providing personalized guidance and insights, The Doyle Group helps clients secure professionals who seamlessly integrate into their teams and culture and drive long-term success.

The Doyle Group understands the importance of exceptional talent in today's dynamic technological landscape. Their dedicated team goes beyond matching candidates with opportunities; they provide ongoing support to both clients and consultants, fostering relationships that thrive over time. They offer a range of services, including project-based, contract-to-hire, direct placement, executive search, offshore, and nearshore solutions. Their consultants specialize in an array of areas, including Digital Solutions, Project and Program Management, Software Development, Data & Analytics, and more.

Whether a business is seeking strategic technology talent or a professional is looking for their next career move, The Doyle Group stands as a reliable partner, committed to delivering lasting value through their expertise and customized services.

Thank you to The Doyle Group for sponsoring our July 2024 meeting!

DMBoK Figure 63 Context Diagram: Data Security

Wed, 26 Jun 2024 13:00:00 GMT

Risk reduction and business growth are the primary drivers of data security activities. Ensuring that an organization’s data is secure reduces risk and adds competitive advantage. Security itself is a valuable asset.

Data security risks are associated with regulatory compliance, fiduciary responsibility for the enterprise and stockholders, reputation, and a legal and moral responsibility to protect the private and sensitive information of employees, business partners, and customers. Organizations can be fined for failure to comply with regulations and contractual obligations. Data breaches can cause a loss of reputation and customer confidence. (See Chapter 2.)

Business growth includes attaining and sustaining operational business goals. Data security issues, breaches, and unwarranted restrictions on employee access to data can directly impact operational success.

The goals of mitigating risks and growing the business can be complementary and mutually supportive if they are integrated into a coherent strategy of information management and protection.

June 2024 Newsletter

Fri, 21 Jun 2024 13:00:00 GMT

June 2024 Newsletter.pdf

DMBOK Figure 62 Sources of Data Security Requirements

Tue, 18 Jun 2024 19:51:48 GMT

Data Security includes the planning, development, and execution of security policies and procedures to provide proper authentication, authorization, access, and auditing of data and information assets. The specifics of data security (which data needs to be protected, for example) differ between industries and countries. Nevertheless, the goal of data security practices is the same: To protect information assets in alignment with privacy and confidentiality regulations, contractual agreements, and business requirements. These requirements come from:

Stakeholders: Organizations must recognize the privacy and confidentiality needs of their stakeholders, including clients, patients, students, citizens, suppliers, or business partners. Everyone in an organization must be a responsible trustee of data about stakeholders.

Government regulations: Government regulations are in place to protect the interests of some stakeholders. Regulations have different goals. Some restrict access to information, while others ensure openness, transparency, and accountability.

Proprietary business concerns: Each organization has proprietary data to protect. An organization’s data provides insight into its customers and, when leveraged effectively, can provide a competitive advantage. If confidential data is stolen or breached, an organization can lose competitive advantage.

Legitimate access needs: When securing data, organizations must also enable legitimate access. Business processes require individuals in certain roles be able to access, use, and maintain data.

Contractual obligations: Contractual and non-disclosure agreements also influence data security requirements. For example, the PCI Standard, an agreement among credit card companies and individual business enterprises, demands that certain types of data be protected in defined ways (e.g., mandatory encryption for customer passwords).

Effective data security policies and procedures ensure that the right people can use and update data in the right way, and that all inappropriate access and update is restricted (Ray, 2012) (see this figure). Understanding and complying with the privacy and confidentiality interests and needs of all stakeholders is in the best interest of every organization. Client, supplier, and constituent relationships all trust in, and depend on, the responsible use of data.

DMBoK Figure 61 SLAs for System and Database Performance

Wed, 12 Jun 2024 13:00:00 GMT

Set Database Performance Levels

System performance, data availability and recovery expectations, and expectations for teams to respond to issues are usually governed through Service Level Agreements (SLAs) between IT data management services organizations and data owners (this figure).

Typically, an SLA will identify the timeframes during which the database is expected to be available for use. Often an SLA will identify a specified maximum allowable execution time for a few application transactions (a mix of complex queries and updates). If the database is not available as agreed to, or if process execution times violate the SLA, the data owners will ask the DBA to identify and remediate the causes of the problem.

DMBoK Figure 60 Log Shipping vs. Mirroring

Wed, 05 Jun 2024 20:33:11 GMT

Data replication means same data is stored on multiple storage devices. In some situations, having duplicate databases is useful, such as in a high-availability environment where spreading the workload among identical databases in different hardware or even data centers can preserve functionality during peak usage times or disasters.

Replication can be active or passive:

Active replication is performed by recreating and storing the same data at every replica from every other replica.

Passive replication involves recreating and storing data on a single primary replica and then transforming its resultant state to other secondary replicas.

Replication has two dimensions of scaling:

Horizontal data scaling has more data replicas.
Vertical data scaling has data replicas located further away in distance geographically.

Multi-master replication, where updates can be submitted to any database node and then ripple through to other servers, is often desired, but increases complexity and cost.

Replication transparency occurs when data is replicated between database servers so that the information remains consistent throughout the database system and users cannot tell or even know which database copy they are using.

The two primary replication patterns are mirroring and log shipping (see this Figure).

In mirroring, updates to the primary database are replicated immediately (relatively speaking) to the secondary database, as part of a two-phase commit process.

In log shipping, a secondary server receives and applies copies of the primary database’s transaction logs at regular intervals.

The choice of replication method depends on how critical the data is, and how important it is that failover to the secondary server be immediate. Mirroring is usually a more expensive option than log shipping. For one secondary server, mirroring is effective; log shipping may be used to update additional secondary servers.

DMBoK Figure 59 Database Organization Spectrum

Wed, 29 May 2024 13:00:00 GMT

Data storage systems provide a way to encapsulate the instructions necessary to put data on disks and manage processing, so developers can simply use instructions to manipulate data. Databases are organized in three general ways: Hierarchical, Relational, and Non-Relational. These classes are not mutually exclusive (see this figure). Some database systems can read and write data organized in relational and non-relational structures. Hierarchical databases can be mapped to relational tables. Flat files with line delimiters can be read as tables with rows, and one or more columns can be defined to describe the row contents.

May 2024 Newsletter

Fri, 24 May 2024 20:17:58 GMT

May 2024 Newsletter.pdf

DMBoK Figure 58 CAP Theorem

Wed, 22 May 2024 18:37:13 GMT

The CAP Theorem (or Brewer’s Theorem) was developed in response to a shift toward more distributed systems (Brewer, 2000). The theorem asserts that a distributed system cannot comply with all parts of ACID at all time. The larger the system, the lower the compliance. A distributed system must instead trade-off between properties.

Consistency: The system must operate as designed and expected at all times.
Availability: The system must be available when requested and must respond to each request.
Partition Tolerance: The system must be able to continue operations during occasions of data loss or partial system failure.

The CAP Theorem states that at most two of the three properties can exist in any shared-data system. This is usually stated with a ‘pick two’ statement, illustrated in this figure.

An interesting use of this theorem drives the Lambda Architecture design discussed in Chapter 14. Lambda Architecture uses two paths for data: a Speed path where availability and partition tolerance are most important, and a Batch path where consistency and availability are most important.

DMBoK Figure 57 Coupling

Wed, 15 May 2024 13:00:00 GMT

Loosely coupled systems require component databases to construct their own federated schema. A user will typically access other component database systems by using a multi-database language, but this removes any levels of location transparency, forcing the user to have direct knowledge of the federated schema. A user imports the data required from other component databases, and integrates it with their own to form a federated schema.

Tightly coupled systems consist of component systems that use independent processes to construct and publish an integrated federated schema, as illustrated in this figure. The same schema can apply to all parts of the federation, with no data replication.

DMBoK Figure 56 Federated Databases

Wed, 08 May 2024 13:00:00 GMT

Federation provisions data without additional persistence or duplication of source data. A federated database system maps multiple autonomous database systems into a single federated database. The constituent databases, sometimes geographically separated, are interconnected via a computer network. They remain autonomous yet participate in a federation to allow partial and controlled sharing of their data. Federation provides an alternative to merging disparate databases. There is no actual data integration in the constituent databases because of data federation; instead, data interoperability manages the view of the federated databases as one large object (see Chapter 8). In contrast, a non-federated database system is an integration of component DBMS’s that are not autonomous; they are controlled, managed and governed by a centralized DBMS.

Federated databases are best for heterogeneous and distributed integration projects such as enterprise information integration, data virtualization, schema matching, and Master Data Management.

Federated architectures differ based on levels of integration with the component database systems and the extent of services offered by the federation. A FDBMS can be categorized as either loosely or tightly coupled.

DMBoK Figure 55 Centralized vs. Distributed

Wed, 01 May 2024 13:00:00 GMT

A database can be classified as either centralized or distributed. A centralized system manages a single database, while a distributed system manages multiple databases on multiple systems. A distributed system’s components can be classified depending on the autonomy of the component systems into two types: federated (autonomous) or non-federated (non-autonomous). This figure illustrates the difference between centralized and distributed.

Centralized databases have all the data in one system in one place. All users come to the one system to access the data. For certain restricted data, centralization can be ideal, but for data that needs to be widely available, centralized databases have risks. For example, if the centralized system is unavailable, there are no other alternatives for accessing the data.

Distributed databases make possible quick access to data over a large number of nodes. Popular distributed database technologies are based on using commodity hardware servers. They are designed to scale out from single servers to thousands of machines, each offering local computation and storage. Rather than rely on hardware to deliver high-availability, the database management software itself is designed to replicate data amongst the servers, thereby delivering a highly available service on top of a cluster of computers. Database management software is also designed to detect and handle failures. While any given computer may fail, the system overall is unlikely to.

Some distributed databases implement a computational paradigm named MapReduce to further improve performance. In MapReduce, the data request is divided into many small fragments of work, each of which may be executed or re-executed on any node in the cluster. In addition, data is co-located on the compute nodes, providing very high aggregate bandwidth across the cluster. Both the filesystem and the application are designed to automatically handle node failures.

DMBoK Figure 54 Context Diagram: Data Storage and Operations

Wed, 24 Apr 2024 13:00:00 GMT

Data Storage and Operations includes the design, implementation, and support of stored data, to maximize its value throughout its lifecycle, from creation/acquisition to disposal. Data Storage and Operations includes two sub-activities:

• Database support focuses on activities related to the data lifecycle, from initial implementation of a database environment, through obtaining, backing up, and purging data. It also includes ensuring the database performs well. Monitoring and tuning are critical to database support.

• Database technology support includes defining technical requirements that will meet organizational needs, defining technical architecture, installing and administering technology, and resolving issues related to technology.

Database administrators (DBAs) play key roles in both aspects of data storage and operations. The role of DBA is the most established and most widely adopted data professional role, and database administration practices are perhaps the most mature of all data management practices. DBAs also play dominant roles in data operations and data security.

April 2024 Newsletter

Fri, 19 Apr 2024 13:00:00 GMT

April 2024 Newsletter.pdf

DMBoK Figure 53 Modeling is Iterative

Wed, 17 Apr 2024 13:00:00 GMT

To build the models, modelers often rely heavily on previous analysis and modeling work. They may study existing data models and databases, refer to published standards, and incorporate any data requirements. After studying these inputs, they start building the model. Modeling is a very iterative process (this Figure). Modelers draft the model, and then return to business professionals and business analysts to clarify terms and business rules. They then update the model and ask more questions (Hoberman, 2014).

DMBoK Figure 52 Supertype and Subtype Relationships

Wed, 10 Apr 2024 13:00:00 GMT

Abstraction is the removal of details in such a way as to broaden applicability to a wide class of situations while preserving the important properties and essential nature from concepts or subjects. An example of abstraction is the Party/Role structure, which can be used to capture how people and organizations play certain roles (e.g., employee and customer). Not all modelers or developers are comfortable with, or have the ability to work with abstraction. The modeler needs to weigh the cost of developing and maintaining an abstract structure versus the amount of rework required if the unabstracted structure needs to be modified in the future (Giles 2011).

Abstraction includes generalization and specialization. Generalization groups the common attributes and relationships of entities into supertype entities, while specialization separates distinguishing attributes within an entity into subtype entities. This specialization is usually based on attribute values within an entity instance.

Subtypes can also be created using roles or classification to separate instances of an entity into groups by function. An example is Party, which can have subtypes of Individual and Organization.

The subtyping relationship implies that all of the properties from the supertype are inherited by the subtype. In the relational example shown in this figure, University and High School are subtypes of School.

Subtyping reduces redundancy on a data model. It also makes it easier to communicate similarities across what otherwise would appear to be distinct and separate entities.

DMBoK Figure 51 Dimensional Physical Data Model

Wed, 03 Apr 2024 13:00:00 GMT

This figure illustrates a dimensional physical data model (usually a star schema, meaning there is one structure for each dimension).

Similar to the relational physical data model, this structure has been modified from its logical counterpart to work with a particular technology to ensure business questions can be answered with simplicity and speed.

A variant of a physical scheme is a Canonical Model, used for data in motion between systems. This model describes the structure of data being passed between systems as packets or messages. When sending data through web services, an Enterprise Service Bus (ESB), or through Enterprise Application Integration (EAI), the canonical model describes what data structure the sending service and any receiving services should use. These structures should be designed to be as generic as possible to enable re-use and simplify interface requirements.

This structure may only be instantiated as a buffer or queue structure on an intermediary messaging system (middleware) to hold message contents temporarily.

DMBoK Figure 50 Relational Physical Data Model

Wed, 27 Mar 2024 13:00:00 GMT

A physical data model (PDM) represents a detailed technical solution, often using the logical data model as a starting point and then adapted to work within a set of hardware, software, and network tools. Physical data models are built for a particular technology. Relational DBMSs, for example, should be designed with the specific capabilities of a database management system in mind (e.g., IBM DB2, UDB, Oracle, Teradata, Sybase, Microsoft SQL Server, or Microsoft Access).

This figure illustrates a relational physical data model. In this data model, School has been denormalized into the Student entity to accommodate a particular technology. Perhaps whenever a Student is accessed, their school information is as well and therefore storing school information with Student is a more performant structure than having two separate structures.

Because the physical data model accommodates technology limitations, structures are often combined (denormalized) to improve retrieval performance, as shown in this example with Student and School.

DMBoK Figure 49 Dimensional Logical Data Model

Wed, 20 Mar 2024 13:00:00 GMT

A dimensional logical data model is in many cases a fully-attributed perspective of the dimensional conceptual data model, as illustrated in this figure. Whereas the logical relational data model captures the business rules of a business process, the logical dimensional captures the business questions to determine the health and performance of a business process.

Admissions Count in this figure is the measure that answers the business questions related to Admissions. The entities surrounding the Admissions provide the context to view Admissions Count at different levels of granularity, such as by Semester and Year.

March 2024 Newsletter

Sat, 16 Mar 2024 17:45:01 GMT

March 2024 Newsletter.pdf

DMBoK Figure 48 Relational Logical Data Model

Sat, 09 Mar 2024 20:54:28 GMT

A logical data model is a detailed representation of data requirements, usually in support of a specific usage context, such as application requirements. Logical data models are still independent of any technology or specific implementation constraints. A logical data model often begins as an extension of a conceptual data model.

In a relational logical data model, the conceptual data model is extended by adding attributes. Attributes are assigned to entities by applying the technique of normalization, as shown in this figure. There is a very strong relationship between each attribute and the primary key of the entity in which it resides. For instance, School Name has a strong relationship to School Code. For example, each value of a School Code brings back at most one value of a School Name.

DMBoK Figure 46 Relational Conceptual Model and Figure 47 Dimensional Conceptual Model

Wed, 06 Mar 2024 19:09:47 GMT

A conceptual data model captures the high-level data requirements as a collection of related concepts. It contains only the basic and critical business entities within a given realm and function, with a description of each entity and the relationships between entities.

For example, if we were to model the relationship between students and a school, as a relational conceptual data model using the IE notation, it might look like Figure 46.

Each School may contain one or many Students, and each Student must come from one School. In addition, each Student may submit one or many Applications, and each Application must be submitted by one Student.

The relationship lines capture business rules on a relational data model. For example, Bob the student can attend County High School or Queens College, but cannot attend both when applying to this particular university. In addition, an application must be submitted by a single student, not two and not zero.

Recall Figure 40, which is reproduced below as Figure 47. This dimensional conceptual data model using the Axis notation, illustrates concepts related to school:

DMBoK Figure 45 Anchor Model

Wed, 28 Feb 2024 14:00:00 GMT

On the Anchor Model, Student, Course, and Attendance are anchors, the gray diamonds represent ties, and the circles represent attributes.

February 2024 Newsletter

Wed, 21 Feb 2024 22:31:45 GMT

February 2024 Newsletter.pdf

DMBoK Figure 44 Data Vault Model

Wed, 21 Feb 2024 14:00:00 GMT

The Data Vault is a detail-oriented, time-based, and uniquely linked set of normalized tables that support one or more functional areas of business. It is a hybrid approach, encompassing the best of breed between third normal form and star schema. Data Vaults are designed specifically to meet the needs of enterprise data warehouses. There are three types of entities: hubs, links, and satellites. The Data Vault design is focused around the functional areas of business with the hub representing the primary key. The links provide transaction integration between the hubs. The satellites provide the context of the hub primary key (Linstedt, 2012).

In this figure, Student and Course are hubs, which represent the main concepts within a subject. Attendance is a link, which relates two hubs to each other. Student Contact, Student Characteristics, and Course Description are satellites that provide the descriptive information on the hub concepts and can support varying types of history. Anchor Modeling is a technique suited for information that changes over time in both structure and content. It provides graphical notation used for conceptual modeling similar to traditional data modeling, with extensions for working with temporal data. Anchor Modeling has four basic modeling concepts: anchors, attributes, ties, and knots. Anchors model entities and events, attributes model properties of anchors, ties model the relationships between anchors, and knots are used to model shared properties, such as states.

DMBoK Figure 42 ORM Model and Figure 43 FCO-IM Model

Wed, 14 Feb 2024 14:00:00 GMT

Object‐Role Modeling (ORM) is a model‐driven engineering approach that starts with typical examples of required information or queries presented in any external formulation familiar to users, and then verbalizes these examples at the conceptual level, in terms of simple facts expressed in a controlled natural language. This language is a restricted version of natural language that is unambiguous, so the semantics are readily grasped by humans; it is also formal, so it can be used to automatically map the structures to lower levels for implementation (Halpin, 2015).

FCO-IM is similar in notation and approach to ORM. The numbers in that figure are references to verbalizations of facts. For example, 2 might refer to several verbalizations including “Student 1234 has first name Bill.”

DMBoK Figure 41 UML Class Model

Wed, 07 Feb 2024 14:00:00 GMT

The Unified Modeling Language (UML) is a graphical language for modeling software. The UML has a variety of notations of which one (the class model) concerns databases. The UML class model specifies classes (entity types) and their relationship types (Blaha, 2013).

Figure 41 illustrates the characteristics of a UML Class Model:

A Class diagram resembles an ER diagram except that the Operations or Methods section is not present in ER.
In ER, the closest equivalent to Operations would be Stored Procedures.
Attribute types (e.g., Date, Minutes) are expressed in the implementable application code language and not in the physical database implementable terminology.
Default values can be optionally shown in the notation.
Access to data is through the class’ exposed interface. Encapsulation or data hiding is based on a ‘localization effect’. A class and the instances that it maintains are exposed through Operations.

The class has Operations or Methods (also called its “behavior”). Class behavior is only loosely connected to business logic because it still needs to be sequenced and timed. In ER terms, the table has stored procedures/triggers.

Class Operations can be:

Public: Externally visible
Internally Visible: Visible to children Objects
Private: Hidden

In comparison, ER Physical models only offer Public access; all data is equally exposed to processes, queries, or manipulations.

DMBoK Figure 40 Axis Notation for Dimensional Models

Wed, 31 Jan 2024 18:17:12 GMT

The concept of dimensional modeling started from a joint research project conducted by General Mills and Dartmouth College in the 1960's. In dimensional models, data is structured to optimize the query and analysis of large amounts of data. In contrast, operational systems that support transaction processing are optimized for fast processing of individual transactions.

Dimensional data models capture business questions focused on a particular business process. The process being measured on this dimensional model is Admissions. Admissions can be viewed by the Zone the student is from, School Name, Semester, and whether the student is receiving financial aid. Navigation can be made from Zone up to Region and Country, from Semester up to Year, and from School Name up to School Level.

The diagramming notation used to build this model - the 'axis notation' - can be a very effective communication tool with those who prefer not to read traditional data modeling syntax.

Both the relational and dimensional conceptual data models can be based on the same business processes (as in this example with Admissions). The difference is in the meaning of the relationships, where on the relational model the relationship lines capture business rules, and on the dimensional model, they capture the navigation paths needed to answer business questions.

DMBoK Figure 38 Dependent and Independent Entity and Figure 39 IE Notation

Wed, 24 Jan 2024 21:22:21 GMT

Identifying vs. Non-Identifying Relationships

An independent entity is one where the primary key contains only attributes that belong to that entity. A dependent entity is one where the primary key contains at least one attribute from another entity. In relational schemes, most notations depict independent entities on the data modeling diagram as rectangles and dependent entities as rectangles with rounded corners

In the student example shown in the Dependent and Independent Entity figure, Student and Course are independent entities and Registration is a dependent entity.

Dependent entities have at least one identifying relationship. An identifying relationship is one where the primary key of the parent (the entity on the one side of the relationship) is migrated as a foreign key to the child's primary key, as can be seen with the relationship from Student to Registration, and from Course to Registration. In non-identifying relationships, the primary key of the parent is migrated as a non-primary foreign key attribute to the child.

Relational

First articulated by Dr. Edward Codd in 1970, relational theory provides a systematic way to organize data so that they reflected their meaning (Codd, 1970). This approach had the additional effect of reducing redundancy in data storage. Codd's insight was that data could most effectively be managed in terms of two-dimensional relations. The term relation was derived from the mathematics (set theory) upon which his approach was based.

The design objectives for the relational model are to have an exact expression of business data and to have one fact in one place (the removal of redundancy). Relational modeling is ideal for the design of operational systems, which require entering information quickly and having it stored accurately.

There are several different kinds of notation to express the association between entities in relational modeling, including Information Engineering (IE), Integration Definition for Information Modeling (IDEF1X), Barker Notation, and Chen Notation. The most common form is IE syntax, with its familiar tridents or 'crows feet' to depict cardinality, see figure IE Notation.

DMBoK Figure 36 Foreign Keys and Figure 37 Attributes

Sat, 13 Jan 2024 19:53:13 GMT

A foreign key is used in physical and sometimes logical relational data modeling schemes to represent a relationship. A foreign key may be created implicitly when a relationship is defined between two entities, depending on the database technology or data modeling tool, and whether the two entities involved have mutual dependencies.

In the example shown in the Foreign Keys figure, Registration contains two foreign keys, Student Number from Student and Course Code from Course. Foreign keys appear in the entity on the many side of the relationship, often called the child entity. Student and Course are parent entities and Registration is the child entity.

An attribute is a property that identifies, describes or measure an entity. Attributes may have domains. The physical correspondent of an attribute in an entity is a column, field, tag, or node in a table, view, document, graph or file.

In data models, attributes are generally depicted as a list within the entity rectangle, as shown in the Attributes figure, where the attributes of the entity Student include Student Number, Student First Name, Student Last Name and Student Birth Date.

January 2024 Newsletter

Sat, 13 Jan 2024 18:14:37 GMT

January 2024 Newsletter.pdf

DMBoK Figure 34 Binary Relationship and Figure 35 Ternary Relationship

Wed, 10 Jan 2024 01:02:57 GMT

An arity of two is also known as binary. A binary relationship, the most common on a traditional data model diagram, involves two entities. The Binary Relationship figure, a UML class diagram, shows that both Student and Course are entities participating in a binary relationship.

An arity of three, known as ternary, is a relationship that includes three entities. An example in fact-based modeling (object -role notation) appears in the Ternary Relationship figure. Here Student can register for a particular Course in a given Semester.

DMBoK Figure 32 Unary Relationship - Hierarchy and Figure 33 Unary Relationship - Network

Wed, 03 Jan 2024 14:00:00 GMT

A unary (also known as a recursive or self-referencing) relationship involves only one entity. A one-to-many recursive relationship describes a hierarchy, whereas a many-to-many relationship describes a network or graph. In a hierarchy, an entity instance has at most one parent (or higher-level entity). In relational modeling, child entities are on the many side of the relationship, with parent entities on the one side of the relationship. In a network, an entity instance can have more than one parent.

For example, a Course can require prerequisites. If, in order to take the Biology Workshop, one would first need to complete the Biology Lecture, the Biology Lecture is the prerequisite for the Biology Workshop. In the following relational data models, which use information engineering notation, one can model this recursive relationship as either a hierarchy or network.

The first example (Figure 32) is a hierarchy and the second (Figure 33) is a network. In the first example, the Biology Workshop requires first taking the Biology Lecture and the Chemistry Lecture. Once the Biology Lecture is chosen as the prerequisite for the Biology Workshop, the Biology Lecture cannot be the prerequisite for any other courses. The second example allows the Biology Lecture to be the prerequisite for other courses as well.

DMBoK Figure 31 Cardinality Symbols

Wed, 20 Dec 2023 14:00:00 GMT

In a relationship between two entities, cardinality captures how many of one entity (entity instances) participates in the relationship with how many of the other entity. Cardinality is represented by the symbols that appear on both ends of a relationship line. Data rules are specified and enforced through cardinality. Without cardinality, the most one can say about a relationship is that two entities are connected in some way.

For cardinality, the choices are simple: zero, one or many. Each side of a relationship can have any combination of zero, one or many ('many' means could be more than 'one'). Specifying zero or one allows us to capture whether or not an entity instance is required in a relationship. Specifying one or many allows us to capture how many of a particular instance participates in a given relationship.

These cardinality symbols are illustrated in the following information engineering example of Student and Course.

The business rules are:

Each Student may attend one or many Courses.
Each Course may be attended by one or many Students.

December 2023 Newsletter

Thu, 14 Dec 2023 19:19:37 GMT

December 2023 Newsletter.pdf

DMBoK Figure 30 Relationships

Wed, 13 Dec 2023 14:00:00 GMT

Relationship

A relationship is an association between entities (Chen, 1976). A relationship captures the high-level interactions between conceptual entities, the detailed interactions between logical entities, and the constraints between physical entities.

Relationship Aliases

The term relationship can go by other names. Relationship aliases can vary based on scheme. In relational schemes the term relationship is often used, dimensional schemes the term navigation path is often used, and in NoSQL schemes terms such as edge or link are used, for example. Relationship aliases can also vary based on level of detail. A relationship at the conceptual and logical levels is called a relationship, but a relationship at the physical level may be called by other names, such as constraint or reference, depending upon the database technology.

Graphic Representation of Relationships

Relationships are shown as lines on the data modeling diagram. This figure is an Information Engineering example.

In this example, the relationship between Student and Course captures the rule that a Student may attend Courses. The relationship between Instructor and Course captures the rule that an Instructor may teach Courses. The symbols on the line (called cardinality) capture the rules in a precise syntax. (These will be explained next week). A relationship is represented through foreign keys in a relational database and through alternative methods for NoSQL databases such as though edges or links.

DMBoK Figure 29 Entities

Wed, 06 Dec 2023 14:00:00 GMT

Graphic Representation of Entities

In data models, entities are generally depicted as rectangles (or rectangles with rounded edges) with their names inside, such as in this figure, where there are 3 entities: Student, Course and Instructor.

Definition of Entities

Entity definitions are essential contributors to the business value of any data model. They are core Metadata. High quality definitions clarify the meaning of business vocabulary and provide rigor to the business rules governing entity relationships. They assist business and IT professionals in making intelligent and application design decisions. High quality data definitions exhibit three essential characteristics:

Clarity: The definition should be easy to read and grasp. Simple, well-written sentences without obscure acronyms or unexplained ambiguous terms such as sometimes or normally.
Accuracy: The definition is a precise and correct description of the entity. Definitions should be reviewed by experts in the relevant business areas to ensure that they are accurate.
Completeness: All of the parts of the definition are present. For example, in defining a code, examples of the code values are included. In defining an identifier, the scope of uniqueness is included in the definition.

DMBoK Figure 28 Context Diagram: Data Modeling and Design

Wed, 29 Nov 2023 14:00:00 GMT

Data Modeling is the process of discovering, analyzing, and scoping data requirements, and then representing and communicating these data requirements in a precise form called the data model. Data modeling is a critical component of data management. The modeling process requires that organizations discover and document how their data fits together. The modeling process itself designs how data fits together (Simsion, 2013). Data models depict and enable an organization to understand its data assets.

There are a number of different schemes used to represent data. The six most commonly used schemes are: Relational, Dimensional, Object-Oriented, Fact-Based, Time-Based, and NoSQL. Models of these schemes exist at three levels of detail: conceptual, logical, and physical. Each model contains a set of components. Examples of components are entities, relationships, facts, keys, and attributes. Once a model is built, it needs to be reviewed and once approved, maintained.

Data models comprise and contain Metadata essential to data consumers. Much of this Metadata uncovered during the data modeling process is essential to other data management functions. For example, definitions for data governance and lineage for data warehousing and analytics.

DMBoK Figure 27 The Data Dependencies of Business Capabilities

Wed, 22 Nov 2023 14:00:00 GMT

Develop a Roadmap

If an enterprise were developed from scratch (free from dependencies on existing processes), and optimal architecture would be based solely on the data required to run the enterprise, priorities would be set by business strategy, and decisions could be made unencumbered by the past. Very few organizations are ever in this state. Even in an ideal situation, data dependencies would quickly arise and need to be managed. A roadmap provides a means to manage these dependencies and make forward-looking decisions. A roadmap helps an organization see trade-offs and formulate a pragmatic plan, aligned with business needs and opportunities, external requirements, and available resources.

A roadmap for Enterprise Data Architecture describes the architecture's 3 - 5 year development path. Together with the business requirements, consideration of actual conditions, and technical assessments, the roadmap describes how the target architecture will become reality. The Enterprise Data Architecture roadmap must be integrated into an overall enterprise architecture roadmap that includes high-level milestones, resources needed, and costs estimations, divided in business capability work streams. The roadmap should be guided by a data management maturity assessment.

Most business capabilities require data as an input; others also produce data on which other business capabilities are dependent. The enterprise architecture and the Enterprise Data Architecture can be formed coherently by resolving this data flow in a chain of dependencies between business capabilities.

A business-data-driven roadmap starts with the business capabilities that are most independent (i.e., have the least dependency from other activities), and ends with those who are most dependent on others. Dealing with each business capability in sequence will follow an overall business data origination order. This figure shows an example chain of dependency, with the lowest dependency at the top. Product Management and Customer Management do not depend on anything else and thus constitute Master Data. The highest dependency items are on the bottom where Customer's Invoice Management depends on Customer Management and Sales Order Management, which in turn depends on two others.

Therefore, the roadmap would ideally advise starting at Product Management and Customer Management capabilities and then resolve each dependency in steps from top to bottom.

November 2023 Newsletter

Thu, 16 Nov 2023 17:11:31 GMT

November 2023 Newsletter.pdf

DMBoK Figure 26 Data Flow Example

Wed, 15 Nov 2023 14:00:00 GMT

Data flows can be documented at different levels of detail: Subject Area, business entity, or even the attribute level. Systems can be represented by network segments, platforms, common application sets, or individual servers. Data flows can be represented by two-dimensional matrices (last week's figure) or in data flow diagrams (this figure).

A matrix gives a clear overview of what data the processes create and use. The benefits of showing the data requirements in a matrix is that it takes into consideration that data does not flow in only one direction; the data exchange between processes are many-to-many in a quite complex way, where any data may appear anywhere. In addition, a matrix can be used to clarify the processes' data acquisition responsibilities and the data dependencies between the processes, which in turn improves the process documentation. Those who prefer working with business capabilities could show this in the same way - just exchanging the processes axis to capabilities. Building such matrices is a long-standing practice in enterprise modeling. IBM introduced this practice in its Business Systems Planning (BSP) method. James Martin later popularized it in his Information Systems Planning (ISP) method during the 1980s.

The data flow in this figure is a traditional high-level data flow diagram depicting what kind of data flows between systems. Such diagrams can be described in many formats and detail levels.

DMBoK Figure 25 Data Flow Depicted in a Matrix

Wed, 08 Nov 2023 14:00:00 GMT

Data flows are a type of data lineage documentation that depicts how data moves through business processes and systems. End-to-end data flows illustrate where the data originated, where it is stored and used, and how it is transformed as it moves inside and between diverse processes and systems. Data lineage analysis can help explain the state of data at a given point in the data flow.

Data flows map and document relationships between data and

Applications within a business process
Data stores or databases in an environment
Network segments (useful for security mapping)
Business roles, depicting which roles have responsibility for creating, updating, using and deleting data (CRUD)
Locations where local differences occur

Data flows can be documented at different levels of detail: Subject Area, business entity, or even the attribute level. Systems can be represented by network segments, platforms, common application sets, or individual servers. Data flows can be represented by two-dimensional matrices (this figure) or in data flow diagrams (next week's figure).

DMBoK Figure 24 Subject Areas Models Diagram Example

Wed, 01 Nov 2023 13:00:00 GMT

This figure depicts three Subject Area diagrams (simplified examples), each containing a Conceptual Data Model with a set of entities. Relationships may cross Subject Area borders; each entity in an enterprise data model should reside in only one Subject Area, but can be related to entities in any other Subject Area.

Hence, the conceptual enterprise data model is built up by the combination of Subject Area models. The enterprise data model can be built using a top-down approach or using a bottom-up approach. The top-down approach means starting with forming the Subject Areas and then populating them with models. When using a bottom-up approach the Subject Area structure is based on existing data models. A combination of the approaches is usually recommended; starting with bottom-up using existing models and completing the enterprise data model by populating the models by delegating Subject Area modeling to projects.

The Subject Area discriminator (i.e., the principles that form the Subject Area structure) must be consistent throughout the enterprise data model. Frequently used subject area discriminator principles include: using normalization rules, dividing Subject Areas from systems portfolios (i.e., funding), forming Subject Areas from data governance structure and data ownership (organizational), using top-level processes (based on the business value chains), or using business capabilities (enterprise architecture-based). The Subject Area structure is usually most effective for Data Architecture work if it is formed using normalization rules. The normalization process will establish the major entities that carry/constitute each Subject Area.

DMBoK Figure 23 Enterprise Data Model

Wed, 25 Oct 2023 13:00:00 GMT

Some organizations create an Enterprise Data Model (EDM) as a stand-alone artifact. In other organizations, it is understood as composed of data models from different perspectives and at different levels of detail, that consistently describe an organization's understanding of data entities, data attributes, and their relationships across the enterprise. An EDM includes both universal (Enterprise-wide Conceptual and Logical Models) and application- or project-specific data models, along with definitions, specifications, mappings and business rules.

Adopting an industry standard model can jumpstart the process of developing an EDM. These models provide a useful guide and references. However, even if an organization starts with a purchased data model, producing enterprise-wide data models requires a significant investment. Work includes defining and documenting an organization's vocabulary, business rules, and business knowledge. Maintaining and enriching an EDM requires an ongoing commitment of time and effort.

An organization that recognizes the need for an EDM must decide how much time and effort it can devote to building and maintaining it. EDMs can be built at different levels of detail, so resource availability will influence initial scope. Over time, as the needs of the enterprise demand, the scope and level of detail captured within an EDM typically expands. Most successful EDMs are built incrementally and iteratively, using layers. This figure shows how different types of models are related and how conceptual models are ultimately linkable to physical application data models. It distinguishes:

A conceptual overview over the enterprise's subject areas
Views of entities and relationships for each subject area
Detailed, partially attributed logical views of these same subject areas
Logical and physical models specific to an application or project

All levels are part of the EDM, and linkages create paths to trace an entity from top to bottom and between models in the same level.

Vertical: Models in each level map to models in other levels. Model lineage is created using these maps. For example, a table or file MobileDevice in a project-specific physical model may link to a MobileDevice entity in the project-specific logical model, a MobileDevice entity in the Product subject area in the Enterprise Logical Model, a Product conceptual entity in the Product Subject Area Model, and to the Product entity in the Enterprise Conceptual Model.
Horizontal: Entities and relationships may appear in multiple models in the same level; entities in logical models centered on one topic may relate to entities in other topics, marked or noted as external to the subject area on the model images. A Product Part entity may appear in the Product subject area models and in the Sales Order, Inventory, and Marketing subject areas, related as external links.

An enterprise data model at all levels is developed suing data modeling techniques.

DMBoK Figure 22 Simplified Zachman Framework

Wed, 18 Oct 2023 13:00:00 GMT

The most well-known enterprise architectural framework, the Zachman Framework, was developed by John A. Zachman in the 1980s. It has continued to evolve. Zachman recognized that in creating buildings, airplanes, enterprises, value chains, projects, or systems, there are many audiences, and each has a different perspective about architecture. He applied this concept to the requirements for different types and levels of architecture within an enterprise.

The Zachman framework is an ontology - the 6 x 6 matrix comprises the complete set of models required to describe an enterprise and the relationships between them. It does not define how to create the models. It simply shows what models should exist.

The two dimensions in the matrix framework are the communication interrogatives (i.e., what, how, where, who, when, why) as columns and the reification transformations (Identification, Definition, Representation, Specification, Configuration, and Instantiation) as rows. The framework classifications are represented by the cells (the intersection between the interrogatives and the transformations). Each cell in the Zachman framework represents a unique type of design artifact.

Communication interrogatives are the fundamental questions that can be asked about any entity. Translated to enterprise architecture, the columns can be understood as follows:

What (the inventory column): Entities used to build the architecture
How (the process column): Activities performed
Where (the distribution column): Business location and technology location
Who (the responsibility column): Roles and organizations
When (the timing column): Intervals, events, cycles, and schedules
Why (the motivation column): Goals, strategies, and means

Reification transformations represent the steps necessary to translate an abstract idea into a concrete instance (an instantiation). These are represented in the rows: planner, owner, designer, builder, implementer and user. Each has a different perspective on the overall process and different problems to solve. These perspectives are depicted as rows. For example, each perspective has a different relation to the What (inventory or data) column:

The executive perspective (business context): Lists of business elements defining scope in identification models.
The business management perspective (business concepts): Clarification of the relationships between business concepts defined by Executive Leaders as Owners in definition models.
The architect perspective (business logic): System logical models detailing system requirements and unconstrained design represented by Architects as Designers in representation models.
The engineer perspective (business physics): Physical models optimizing the design for implementation for specific use under the constraints of specific technology, people, costs, and timeframes specified by Engineers as Builders in specifications models.
The technician perspective (component assemblies): A technology-specific, out-of-context view of how components are assembled and operated configured by Technicians as Implementers in configuration models.
The user perspective (operations classes): Actual functioning instances used by Workers as Participants. There are no models in this perspective.

As noted previously, each cell in the Zachman Framework represents a unique type of design artifact, defined by the intersection of its row and column. Each artifact represents how the specific perspective answers the fundamental questions.

October 2023 Newsletter

Fri, 13 Oct 2023 00:14:56 GMT

October 2023 Newsletter.pdf

Welcome New Members!

Thu, 12 Oct 2023 03:54:54 GMT

Welcome New Members!

We'd like to welcome our 20 new Professional Members who have joined the chapter in Q2 & Q2 2023. We're thrilled you're here and hope you are enjoying all the perks of membership!

Allen A
Alyssa S
Brandon A
Devan D
Emilie A
Gisella B
Gurvirender T
Indira K
Jason V
Jon S
Joseph B
Lara S
Lee G
Linda G
Martin P
May C
Michael B
Michael P
Pete G
Zack R

We also warmly welcome our 60 new Guest Members! We're excited you're here and hope you explore and find a way to connect with our community at an upcoming event.

DMBoK Figure 21 Context Diagram : Data Architecture

Wed, 11 Oct 2023 13:30:00 GMT

Primary Data Architecture outcomes include:

Data storage and processing requirements
Designs of structures and plans that meet the current and long-term data requirements of the enterprise

Architects seek to design in a way that brings value to the organization. this values comes through an optimal technical footprint, operational and project efficiencies, and the increased ability of the organization to use its data. to get there requires good design, planning, and the ability to ensure that the designs and plans are executed effectively.

To reach these goals, Data Architects define and maintain specifications that:

Define the current state of data in the organization
Provide a standard business vocabulary for data and components
Align Data Architecture with enterprise strategy and business architecture
Express strategic data requirements
Outline high-level integrated designs to meet these requirements
Integrate with overall enterprise architecture roadmap

An overall Data Architecture practice includes:

Using Data Architecture artifacts (master blueprints) to define data requirements, guide data integration, control data assets, and align data investments with business strategy
Collaborating with, learning from and influencing various stakeholders that are engaged with improving the business or IT systems development
Using Data Architecture to establish the semantics of an enterprise, via a common business vocabulary

Guest Bloggers Needed

Sat, 07 Oct 2023 19:25:59 GMT

DAMA-RMC is looking for guest bloggers to be featured on our website, and in our newsletters and social media posts. This is a great opportunity to grow your network and reach thousands of new contacts sharing your data knowledge and expertise. Interested bloggers can reach out to Communications@damarmc.org.

Details for submission and publishing are as follows:

Guest bloggers will be featured in the members only section of our website.
The guest blog and the blogger's contact information can be featured in our emailed monthly newsletter, which reaches 600+ members (pending delivery and approval by the time newsletters release later in the 2nd week of each month)
The guest blog and the blogger's contact information will be featured in social media posts on Linked In and Twitter (reaches multiple 100s of 1000s of people) as well as a monthly Linked In newsletter.
The guest blog and the blogger's contact information can be featured in DAMA-RMC Linked In newsletter (pending delivery and approval by the time newsletters release earlier in the 3rd week of each month)
There is no compensation for guest blogging. There is a trade in promotion of you and your contact information to our network. An additional benefit could be an invitation to speak to our chapter in your area of expertise.
When considering topics to submit, please refer to the Knowledge Areas in the DAMA Wheel below and additional DMBoK topics of data management, data handling ethics, big data, and data science.
There is no character limit and longer blogs can certainly be published in multiple parts across weeks or months.
Blogs should not be vendor specific. Case studies can mention vendors and should not be a sales pitch of vendor specific features.
Submissions should be emailed to Communications@damarmc.org, by the 1st day of each month to be considered and then reviewed/approved by the DAMA board before release.
Submissions should be new content from the blogger and not made available to the general public anywhere else for at least one month after DAMA publishing.

For a range of submission topics, please refer to the DAMA Wheel:

Thank you for your interest in being a guest blogger for DAMA-RMC.

DMBoK Figure 20 Data Issue Escalation Path

Wed, 04 Oct 2023 13:30:00 GMT

Issue Management is the process for identifying, quantifying, prioritizing, and resolving data governance-related issues, including:

Authority: Questions regarding decision rights and procedures
Change management escalations: Issues arising from the change management process
Compliance: Issues with meeting compliance requirements
Conflicts: Conflicting policies, procedures, business rules, names, definitions, standards, architecture, data ownerships and conflicting stakeholder interests in data and information
Conformance: Issue related to conformance to policies, standards, architecture, and procedures
Contracts: Negotiation and review of data sharing agreements, buying and selling data, and cloud storage
Data security and identity: Privacy and confidentiality issues, including breach investigations
Data quality: Detection and resolution of data quality issues, including disasters or security breaches

Many issues can arise locally in Data Stewardship teams. Issues requiring communication and / or escalation must be logged, and may be escalated to the Data Stewardship teams, or higher to the DGC, as shown in this figure. A Data Governance scorecard can be used to identify trends related to issues, such as where within the organization they occur, what their root causes are, etc. Issues that cannot be resolved by the DGC should be escalated to corporate governance and / or management.

Data governance requires control mechanisms and procedures for:

Identifying, capturing, logging, tracking and updating issues
Assignment and tracking of action items
Documenting stakeholder viewpoints and resolution alternatives
Determining, documenting, and communicating issue resolutions
Facilitating objective, neutral discussions where all viewpoints are heard
Escalating issues to higher levels of authority

Data issue management is very important. It builds credibility for the DG team, has direct, positive effects on data consumers, and relieves the burden on production support teams. Solving issue management requires control mechanisms that demonstrate the work effort and impact of resolution.

DMBoK Figure 19 An Example of an Operating Framework

Wed, 27 Sep 2023 13:30:00 GMT

While developing a basic definition of Data Governance (DG) is easy, creating an operating model that an organization will adopt can be difficult. Consider these areas when constructing an organization's operating model:

Value of data to the organization: If an organization sells data, obviously DG has a huge business impact. Organizations that use data as a crucial commodity (e.g. Facebook, Amazon) will need an operating model that reflects the role of the data. For organizations where data is an operational lubricant, the form of DG will be less intense.
Business model: Decentralized business vs. centralized, local vs. international, etc. are factors that influence how business occurs, and therefore, how the DG operating model is defined. Links with specific IT strategy, Data Architecture, and application integration functions should be reflected in the target operating framework design (per previous post on DMBoK Figure 16).
Cultural factors: Such as acceptance of discipline and adaptability to change. Some organizations will resist the imposition of governance by policy and principle. Governance strategy will need to advocate for an operating model that fits the organizational culture, while still progressing change.
Impact of regulation: Highly regulated organizations will have a different mindset and operating model of DG than those less regulated. There may be links to the Risk Management group or Legal as well.

Layers of data governance are often part of the solution. This means determining where accountability should reside for stewardship activities, who owns the data, etc. The operating model also defines the interaction between the governance organization and the people responsible for data management projects or initiatives, the engagement of change management activities to introduce this new program, and the model for issue management resolution pathways through governance. This figure shows an example of an operating framework. The example is illustrative. This kind of artifact must be customized to meet the needs of a specific organization.

DMBoK Figure 18 CDO Organizational Touch Points

Wed, 20 Sep 2023 13:30:00 GMT

Part of alignment includes developing organizational touchpoints for Data Governance work. The CDO Organizational Touch Points figure illustrates examples of touch points that support alignment and cohesiveness of an enterprise data governance and data management approach in areas outside the direct authority of the Chief Data Officer.

Procurements and Contracts: The CDO works with a Vendor/Partner Management or Procurement to develop and enforce standard contract language vis-a-vis data management contracts. These could include Data-as-a-Service (DaaS) and cloud-related procurements, other outsourcing arrangements, third-party development efforts, or content acquisition / licensing deals, and possibly data-centric IT tools acquisitions and upgrades.
Budget and Funding: If the CDO is not directly in control of all data acquisition-related budgets, then the office can be a focal point for preventing duplicate efforts and ensuring optimizations of acquired data assets.
Regulatory Compliance: The CDO understands and works within required local, national, and international regulatory environments, and how these impact the organization and their data management activities. Ongoing monitoring is performed to identify and track new and potential impacts and requirements.
SDLC / development framework: The data governance program identifies control points where enterprise policies, processes, and standards can be developed in the system or application development lifecycles.

The touch points that the CDO influences support the organization's cohesiveness in managing its data, therefore, increasing its nimbleness to use its data. In essence, this is a vision of how DG will be perceived by the organization.

September 2023 Newsletter

Thu, 14 Sep 2023 16:00:00 GMT

September 2023 Newsletter.pdf

DMBoK Figure 17 Enterprise Data Governance Operating Framework Examples

Wed, 13 Sep 2023 13:30:00 GMT

In a centralized model, one Data Governance organization oversees all activities in all subject areas. In a replicated model, the same DG operating model and standards are adopted by each business unit. In a federated model, one Data Governance organization coordinates with multiple Business Units to maintain consistent definitions and standards.

The most formal and mature data management operating model is a centralized one. Here everything is owned by the Data Management Organization. Those involved in governing and managing data report directly to a data management leader who is responsible for Governance, Stewardship, Metadata Management, Data Quality Management, Master and Reference Data Management, Data Architecture, Business Analysis, etc.

DMBoK Figure 16 Data Governance Organization Parts

Wed, 06 Sep 2023 13:30:00 GMT

The core word in governance is govern. Data governance can be understood in terms of political governance. It includes legislative-like functions (defining policies, standards and the Enterprise Data Architecture), judicial-like functions (issue management and escalation), and executive functions (protecting and serving, administrative responsibilities). To better manage risk, most organizations adopt a representative form of data governance, so that all stakeholders can be heard.

Each organization should adopt a governance model that supports its business strategy and it likely to succeed within its own cultural context. Organizations should also be prepared to evolve that model to meet new challenges. Models differ with respect to their organizational structure, level of formality, and approach to decision-making. Some models are centrally organized, while others are distributed.

Data governance organizations may also have multiple layers to address concerns at different levels within an enterprise - local, divisional, and enterprise-wide. The work of governance is often divided among multiple committees, each with a purpose and level of oversight different from others.

This Figure represents a generic data governance model, with activities at different levels within the organization (vertical axis), as well as separation of governance responsibilities within organizational functions and between technical (IT) and business areas.

DMBoK Figure 15 Data Governance and Data Management

Wed, 30 Aug 2023 14:30:00 GMT

Just as an auditor controls financial processes but does not actually executive financial management, data governance ensures data is properly managed without directly executing data management. Data governance represents an inherent separation of duty between oversight and execution.

A data-centric organization values data as an asset and manages data through all phases of its lifecycle, including project development and ongoing operations. To become data-centric, and organization must change the way it translates strategy into action. Data is no longer treated as a by-product of process and applications. Ensuring data is of high quality is a goal of business processes. As organizations strive to make decisions based on insights gained from analytics, effective data management becomes a very high priority.

People tend to conflate data and information technology. To become data-centric, organizations need to think differently and recognize that managing data is different from managing IT. This shift is not easy. Existing culture, with its internal politics, ambiguity about ownership, budgetary competition, and legacy systems, can be a huge obstacle to establishing an enterprise vision of data governance and data management.

While each organization needs to evolve its own principles, those that seek to get more value from their data are likely to share the following:

Data should be managed as a corporate asset
Data management best practices should be incented across the organization
Enterprise data strategy must be directly aligned with overall business strategy
Data management processes should be continuously improved

DMBoK Figure 14 Context Diagram: Data Governance and Stewardship

Wed, 23 Aug 2023 14:30:00 GMT

Data Governance (DG) is defined as the exercise of authority and control (planning, monitoring, and enforcement) over the management of data assets. All organizations make decisions about data, regardless of whether they have a formal Data Governance function. Those that establish a formal Data Governance program exercise authority and control with greater intentionality (Seiner, 2014). Such organizations are better able to increase the value they get from their data assets.

The Data Governance function guides all other data management functions. The purpose of Data Governance is to ensure that data is managed properly, according to policies and best practices (Ladley, 2012). While the driver of data management overall is to ensure an organization gets value out of its data, Data Governance focuses on how decisions are made about data and how people and processes are expected to behave in relation to data. The scope and focus of a particular data governance program will depend upon organizational needs, but most programs include:

Strategy: Defining, communicating, and driving execution of Data Strategy and Data Governance Strategy
Policy: Setting and enforcing policies related to data and Metadata management, access, usage, security, and quality
Standards and quality: Setting and enforcing Data Quality and Data Architecture standards
Oversight: Providing hands-on observation, audit, and correction in key areas of quality, policy and data management (often referred to as stewardship)
Compliance: Ensuring the organization can meet data-related regulatory compliance requirements
Issue management: Identifying, defining, escalating, and resolving issues related to data security, data access, data quality, regulatory compliance, data ownership, policy, standards, terminology, or data governance procedures
Data management projects: Sponsoring efforts to improve data management practices
Data asset valuation: Setting standards and processes to consistently define the business value of data assets

To accomplish these goals, a Data Governance program will develop policies and procedures, cultivate data stewardship practices at multiple levels within the organization, and engage in organizational change management efforts that actively communicate to the organization the benefits of improved data governance and the behaviors necessary to successfully manage data as an asset.

For most organizations, adopting formal Data Governance requires the support of organizational change management, as well as sponsorship from a C-Level executive, such as Chief Risk Officer, Chief Financial Officer, or Chief Data Officer.

Join Our 12-Week Virtual #CDMP Study Group!

Thu, 17 Aug 2023 22:57:01 GMT

Are you ready to take your career to the next level? We're thrilled to announce the launch of our 12-week virtual Certified Data Management Professional (CDMP) Study Group, designed to help you succeed in the world of data management. Learn more about the CDMP Exam HERE.

Start Date: Week of September 4th
Meeting Time: Thursday evenings from 6:00 PM to 7:00 PM
Location: Virtual (Online Platform)

What you can expect:
• Engaging weekly study sessions to cover CDMP exam topics.
• Interactive discussions and Q&A sessions to enhance your understanding.
• A 1/2 day virtual exam prep deep dive the last week in November.
• "Pay if you Pass" (PIYP) virtual exam event on November 2nd.

Informational Sessions:
Dates: August (24th and 28th)
Where: Online (Email for meeting registration info)
Purpose: Learn about the study group and CDMP exam, get your questions answered.

How to Get Involved:

To learn more, register, or secure your spot for this enriching opportunity, simply send an email to: ProfessionalDevelopmentVP@damarmc.org. Hurry, spaces are limited!

Don't miss out on this chance to boost your career prospects and become a certified data management professional. We look forward to embarking on this exciting journey with you!

Don't forget to order your essential study resource, the #DMBOK (Data Management Body of Knowledge), to ensure you're fully prepared to excel in both your studies and the CDMP exam. Order through DAMA-RMC at a discounted rate HERE.

And remember you must be a DAMA-RMC professional member to take advantage of this amazing opportunity. Don’t delay, join today HERE.

Stay tuned for more updates and detailed schedules.

DMBoK Figure 13 Ethical Risk Model for Sampling Projects

Wed, 16 Aug 2023 14:30:00 GMT

Projects that use personal data should have a disciplined approach to the use of that data. They should account for:

How they select their population for study (arrow 1)
How data will be captured (arrow 2)
What activities analytics will focus on (arrow 3)
How the results will be made accessible (arrow 4)

Within each area of consideration, they should address potential ethical risks, with a particular focus on possible negative effects on customers or citizens.

A risk model can be used to determine whether to execute the project. It will also influence how to execute the project. For example, the data will be made anonymous, the private information removed from the file, the security on the files tightened or confirmed, and a review of the local and other applicable privacy law reviewed with legal. Dropping customers may not be permitted under law if the organization is a monopoly in a jurisdiction, and citizens have no other provider options such as energy or water.

Because data analytics projects are complex, people may not see the ethical challenges. Organizations need to actively identify potential risks. They also need to protect whistleblowers who do see risks and raise concerns. Automated monitoring is not sufficient protection from unethical activities. People - the analysts themselves - need to reflect on possible bias. Cultural norms and ethics in the workplace influence corporate behavior - learn and use the ethical risk model. DAMA International encourages data professionals to take a professional stand, and present the risk situation to business leaders who may not have recognized the implications of particular uses of data and these implications in their work.

August 2023 Newsletter

Thu, 10 Aug 2023 22:37:53 GMT

August 2023 Newsletter.pdf

DMBoK Figure 12 Context Diagram: Data Handling Ethics

Wed, 09 Aug 2023 14:30:00 GMT

Defined simply, ethics are principles of behavior based on ideas of right and wrong. Ethical principles often focus on ideas such as fairness, respect, responsibility, integrity, quality, reliability, transparency, and trust. Data handling ethics are concerned with how to procure, store, manage, use, and dispose of data in ways that are aligned with ethical principles. Handling data in an ethical manner is necessary to the long-term success of any organization that wants to get value from its data. Unethical #datahandling can result in the loss of reputation and customers, because it puts at risk people whose data is exposed. In some cases, unethical practices are also illegal. Ultimately, for #datamanagement professionals and the organizations for which they work, data ethics are a matter of social responsibility.

The ethics of data handling are complex, but they center on several core concepts:

Impact on people: Because data represents characteristics of individuals and is used to make decisions that affect people's lives, there is an imperative to manage its quality and reliability.
Potential for misuse: Misusing data can negatively affect people and organizations, so there is an ethical imperative to prevent the misuse of data.
Economic value of data: Data has economic value. Ethics of #dataownership should determine how that value can be accessed and by whom.

Organizations protect data based largely on lows and regulatory requirements. Nevertheless, because data represents people (customers, employees, patients, vendors, etc.), data management professionals should recognize that there are ethical (as well as legal) reasons to protect data and ensure it is not misused. Even data that does not directly represent individuals can still be used to make decisions that affect people's lives.

There is an ethical imperative not only to protect data, but also manage its quality. People making decisions, as well as those impacted by decisions, expect data to be complete and accurate. From both a business and a technical perspective, data management professionals have an ethical responsibility to manage data in a way that reduces risk that it may misrepresent, be misused, or be misunderstood. This responsibility extends across the data lifecycle, from creation to destruction of data.

Unfortunately, many organizations fail to recognize and respond to the ethical obligations inherent in data management. They may adopt a traditional technical perspective and profess not to understand the data; or they assume that if they follow the letter of the law, they have no risk related to data handling. This is a dangerous assumption.

The data environment is evolving rapidly. Organizations are using data in ways they would not have imagined even a few years ago. While laws codify some ethical principles, legislation cannot keep up with the risks associated with evolution of the data environment. Organizations must recognize and respond to their ethical obligation to protect data entrusted to them by fostering and sustaining a culture that values the ethical handling of information.

DMBoK Figure 11 DAMA Wheel Evolved

Wed, 02 Aug 2023 14:30:00 GMT

The DAMA Data Management Framework can also be depicted as an evolution of the DAMA Wheel, with core activities surrounded by lifecycle and usage activities, contained within the structure of governance.

Core activities, including Metadata Management, Data Quality Management, and data structure definition (architecture) are at the center of the framework.

Lifecycle management activities may be defined from a planning perspective (risk management, modeling, data design, Reference Data Management) and an enablement perspective (Master Data Management, data technology development, data integration and interoperability, data warehousing, and data storage and operations.)

Usages emerge from the lifecycle management activities: Master data usage, Document and content management, Business Intelligence, Data Science, predictive analytics, data visualization. Many of these create more data by enhancing or developing insights about existing data. Opportunities for data monetization may be identified as uses of data.

Data governance activities provide oversight and containment, through strategy, principles, policy, and stewardship. They enable consistency through data classification and data valuation.

The intention in presenting different visual depictions of the DAMA Data Management Framework is to provide additional perspective and to open discussion about how to apply the concepts presented in the DMBoK. As the importance of data management grows, such frameworks become useful communications tolls both within the data management community and between the data management community and our stakeholders.

DMBoK Figure 10 DAMA Data Management Function Framework

Wed, 26 Jul 2023 14:30:00 GMT

A third alternative to the DAMA Wheel is depicted in the DAMA Data Management Function Framework. This also draws on architectural concepts to propose a set of relationships between the DAMA Knowledge Areas. It provides additional detail about the content of some Knowledge Areas in order to clarify these relationships.

The framework starts with the guiding purpose of data management: To enable organizations to get value from their data assets as they do from other assets. Deriving value requires lifecycle management, so data management functions related to the data lifecycle are depicted in the center of the diagram. These include planning and designing for reliable, high-quality data; establishing processes and functions through which data can be enabled for use and also maintained; and, finally, using the data in various types of analysis and through those processes, enhancing its value.

The lifecycle management section depicts the data management design and operational functions (modeling, architecture, storage and operations, etc.) that are required to support traditional uses of data (Business Intelligence, document and content management). It also recognizes emerging data management functions (Big Data storage) that support emerging uses of data (data Science, predictive analytics, etc.) In cases where data is truly managed as an asset, organizations may be able to get direct value from their data by selling it to other organizations (data monetization).

Organizations that focus only on direct lifecycle functions will not get as much value from their data as those that support the data lifecycle through foundational and oversight activities. Foundational activities, like data risk management, Metadata, and Data Quality management, span the data lifecycle. They enable better design decisions and make data easier to use. If these are executed well, data is less expensive to maintain, data consumers have more confidence in it, and the opportunities for using it expand.

To successfully support data production and use and to ensure that foundational activities are executed with discipline, many organizations establish oversight in the form of data governance. A data governance program enables an organization to be data-driven, by putting in place the strategy and supporting principles, policies, and stewardship practices that ensure the organization recognizes and acts on opportunities to get value from its data. A data governance program should also engage in organizational change management activities to educate the organization and encourage behaviors that enable strategic uses of data. Thus, the necessity of culture change spans the breath of data governance responsibilities, especially as an organization matures its data practices.

DMBoK Figure 9 DAMA Functional Area Dependencies

Wed, 19 Jul 2023 14:30:00 GMT

Aiken's pyramid describes how organizations evolve toward better data management practices. Another way to look at the DAMA Knowledge Areas is to explore the dependencies between them. Developed by Sue Geuens, the Functional Area Dependencies framework recognizes that Business Intelligence and Analytic functions have dependencies on all other data management functions. They depend directly on Master Data and Data Warehouse solutions. But those, in turn, are dependent on feeding systems and applications. Reliable Data Quality, data design, and data interoperability practices are at the foundation of reliable systems and applications. In addition, Data Governance, which within this model includes Metadata Management, data security, Data Architecture and Reference Data Management, provides a foundation on which all other functions are dependent.

July 2023 Newsletter

Thu, 13 Jul 2023 16:42:36 GMT

July 2023 Newsletter.pdf

DMBoK Figure 8 Pyramid (Aiken)

Wed, 12 Jul 2023 21:09:03 GMT

Peter Aiken's framework uses the DMBoK functional areas to describe the situation in which many organizations find themselves. An organization can use it to define a way forward to a state where they have reliable data and processes to support strategic business goals. In trying to reach this goal, many organizations go through a similar logical progression of steps.

Phase 1: The organization purchases an application that includes database capabilities. This means the organization has a starting point for #datamodeling / design, #datastorage, and #datasecurity (e.g., let some people in and keep others out). To get the system functioning within their environment and with their data requires work on integration and interoperability.

Phase 2: Once they start using the application, they will find challenges with the quality of their data. But getting to higher quality data depends on reliable #Metadata and consistent #DataArchitecture. These provide clarity on how data from different systems works together.

Phase 3: Disciplined practices for managing #DataQuality, Metadata, and architecture require #DataGovernance that provides structural support for data management activities. Data Governance also enables execution of strategic initiatives, such as Document and #ContentManagement, #ReferenceDataManagement, #MasterDataManagement, #DataWarehousing, and #BusinessIntelligence, which fully enable the advanced practices within the golden pyramid.

Phase 4: The organization leverages the benefits of well-managed data and advances its analytic capabilities.

Aiken's pyramid draws from the DAMA Wheel, but also informs it by showing the relation between the Knowledge Areas. They are not all interchangeable; they have various kinds of interdependencies. The Pyramid framework has two drivers. First, the idea of building on a foundation, using components that need to be in the right places to support each other. Second, the somewhat contradictory idea that these may be put in place in an arbitrary order.

DMBoK Figure 7 Knowledge Area Context Diagrams

Wed, 05 Jul 2023 14:30:00 GMT

The Knowledge Area (KA) Context Diagrams describe the detail of the KAs, including detail related to people processes & technology. They are based on the concept of a SIPOC diagram used for product management (Suppliers, Inputs, Processes, Outputs, & Consumers). Context Diagrams put activities at the center, since they produce the deliverables that meet the requirements of stakeholders.

Each context diagram begins with the KA's definition and goals. activities that drive the goals (Center) are classified into four phases: Plan (P), Develop (D), Operate (O), & Control (C). On the left side (flowing into activities) are the Inputs & Suppliers. On the right side (flowing out of the activities) are Deliverables & Consumers. Participants are listed below the Activities. On the bottom are Tools, Techniques, & Metrics that influence aspects of the KA.

Lists in the context diagram are illustrative, not exhaustive. Items will apply differently to different organizations. The high-level role lists include only the most important roles. Each organization can adapt this pattern to address its own needs.

The component pieces of the context diagram:

1. Definition - concisely define the KA
2. Goals - purpose of KA & fundamental principles that guide performance of activities within KA
3. Activities - actions/tasks required to meet goals of KA
a. (P) Planning Activities - set strategic/tactical course for meeting data management goals
b. (D) Development Activities - organized around Software Development Life Cycle (SDLC (analysis, design, build, test, prepare, deploy)
c. (C) Control Activities - ensure ongoing quality of data & the integrity, reliability, & security of systems where data is accessed & used
d. (O) Operational Activities - support use, maintenance & enhancement of systems/processes where data is accessed/used
4. Inputs - tangible things that each KA requires to initiate its activities
5. Deliverables - outputs of KA activities, tangible things each function is responsible for producing
6. Roles & Responsibilities - describe how individuals/teams contribute to activities within KA
7. Suppliers - people responsible to provide/enable access to activity inputs
8. Consumers - those that benefit directly from primary deliverables by the data management activities
9. Participants - people that perform, manage the performance of, or approve the KA activities
10. Tools - applications/other technologies that enable KA goals
11. Techniques - methods/procedures used to perform activities & produce deliverables in the KA
12. Metrics - measurement or performance, progress, quality, or efficiency evaluation standards

DMBoK Figure 6 Environmental Factors Hexagon

Wed, 28 Jun 2023 14:30:00 GMT

The Environmental Factors Hexagon shows the relationship between people, process, and technology and provides a key for reading the DMBoK context diagrams. It puts goals and principles at the center, since these provide guidance for how people should execute activities and effectively use the tools required for successful data management.

DMBoK Figure 4 Amsterdam Information Model

Wed, 21 Jun 2023 14:30:00 GMT

The Amsterdam Information Model (AIM), like the Strategic Alignment Model (SAM), takes a strategic perspective on business and IT alignment. Known as the 9-cell, it recognizes a middle layer that focuses on structure and tactics, including planning and architecture. More ever, it recognizes the necessity of information communication (expressed as the information governance and data quality pillar in the AIM Model).

The creators of both the SAM and the AIM frameworks describe in detail the relation between the components, from both the horizontal (Business/IT strategy) and vertical (Business Strategy / Business Operations) perspective.

June 2023 Newsletter

Tue, 20 Jun 2023 16:11:47 GMT

June 2023 Newsletter.pdf

DMBoK Figure 3 Strategic Alignment Model

Wed, 14 Jun 2023 18:00:00 GMT

The Strategic Alignment Model (SAM) abstracts the fundamental drivers for any approach to #datamanagement. At its center is the relationship between data and information. Information is most often associated with #businessstrategy and the operational use of data. Data is associated with information technology and processes which support physical management of systems that make data accessible for use. Surrounding this concept are the four fundamental domains of strategic choice: business strategy, information technology strategy, organizational infrastructure and processes, and information technology infrastructure and processes.

The fully articulated SAM is more complex than is illustrated. Each of the corner hexagons has its own underlying dimensions. For example, within Business and IT strategy, there is a need to account for scope, competencies and governance. Operations must account for infrastructure, processes and skills. The relationship between the pieces help an organization understand both the strategic fit of the different components and functional integration of the pieces. Even the high-level depiction of the model is useful in understanding the organizational factors that influence decisions and data and data management.

DMBoK Figure 2 Data Lifecycle Key Activities

Mon, 05 Jun 2023 15:30:00 GMT

The important implications of the Key Activities of the Data Lifecycle from the Data Management Book of Knowledge are:

* Creation and usage are the most critical points in the data lifecycle
* Data Quality must be managed throughout the data lifecycle
* Metadata Quality must be managed through the data lifecycle
* Data Security must be managed through the data lifecycle
* Data Management efforts should focus on the most critical data

DMBoK Figure 1 Data Management Principles

Tue, 30 May 2023 18:00:00 GMT

The key Data Management Principles from the Data Management Book of Knowledge are:

* Data is an asset with unique properties.
* The value of data can and should be expressed in economic terms.
* Managing data means managing the quality of data.
* It takes Metadata to manage data.
* It takes planning to manage data.
* Data management is cross-functional; it requires a range of skills and expertise.
* Data management requires an enterprise perspective.
* Data management must account for a range of perspectives.
* Data management is lifecycle management.
* Different types of data have different lifecycle characteristics.
* Managing data includes managing the risks associated with data.
* Data management requirements must drive Information Technology decisions.
* Effective data management requires leadership commitment.

DMBoK Figure 5 DAMA-DMBoK2 Data Management Framework (The DAMA Wheel)

Mon, 15 May 2023 21:13:02 GMT

The DAMA Wheel defines the Data Management Knowledge Areas. It places data governance at the center of data management activities, since governance is required for consistency within and balance between the functions. The other Knowledge Areas (Data Architecture, Data Modeling, etc.) are balanced around the Wheel. They are all necessary parts of a mature data management function, but they may be implemented at different times, depending upon the requirements of the organization. These Knowledge Areas are the focus of Chapters 3 - 13 of the DMBoK 2nd Edition.

May 2023 Newsletter

Mon, 15 May 2023 18:27:44 GMT

May 2023 Newsletter.pdf

April 2023 Newsletter

Thu, 13 Apr 2023 16:18:32 GMT

April 2023 Newsletter.pdf

Welcome New Members!

Wed, 12 Apr 2023 18:45:25 GMT

We'd like to welcome our new Professional Members who have joined the chapter in Q4 2022 and Q1 2023. We're excited you're here and hope you are enjoying all the perks of membership!