Development Process

The Data Analytics Development Process is a structured approach to developing data analytics projects. It consists of several key steps that guide the project from initiation to completion. The process is designed to ensure that projects are completed efficiently, effectively, and in alignment with business goals. The following sections outline the key steps in the Data Analytics Development Process.

Key Steps in the Data Analytics Project Process

Project Request: The process begins with a project request from the business team or individual. This request should include a description of the project, its goals, and any relevant information.
Request Evaluation: The data analytics team evaluates the request to determine if it is a legitimate project or a support request. Legitimate projects are further categorized into novel or routine projects.
Project Definition: This step involves defining the scope of the project, identifying stakeholders, and determining the resources needed. It may include exploratory data analysis (EDA), defining data sources, and testing modeling approaches.
Project Type: The project is classified as either a novel or routine project. Novel projects are gated and require stakeholder approval, while routine projects can proceed without formal approval.
Project Initiation: The project is initiated by the data analytics team, often at the request of a business individual or team. This includes creating a project plan, identifying resources, and determining the timeline.
Project Execution: The project is executed by the data analytics team with assistance from subject matter experts (SMEs) and other stakeholders. This includes creating data pipelines, models, reports, dashboards, and documentation.
Project Closure: Once the project is completed, it is closed out. This includes finalizing documentation, conducting a project review, and ensuring that all deliverables are met.
Project Review: The project is reviewed to ensure it meets stakeholder requirements and was completed on time and within budget. Not all projects are reviewed, but it is a good practice for large or complex projects.

Now we have a project, How do we develop a solution?

The development of a data analytics solution involves several key steps, including data preparation, model development, and deployment. The following sections outline the key steps in the Data Analytics Development Process.

Advanced Analytics Development Process

Advanced analytics projects are diverse and can include but are not limited to:

exploratory data analysis (EDA) projects
machine learning (ML) projects
User Interface (UI) development projects

Exploratory Data Analysis (EDA) Projects

Data Preparation: This step involves preparing the data for analysis. This includes data cleaning, data transformation, and data integration. The goal is to ensure that the data is in a format that can be easily analyzed and that it is of high quality. In some cases this may include the creation of a data pipeline to ingest and transform the data.
Exploratory Data Analysis (EDA): This step involves analyzing the data to understand its structure, relationships, and patterns. This includes visualizing the data, identifying trends, and discovering insights. The goal is to gain a deeper understanding of the data and identify potential areas for further analysis.

ML Projects

Data Preparation: This step involves preparing the data for analysis. This includes data cleaning, data transformation, and data integration. The goal is to ensure that the data is in a format that can be easily analyzed and that it is of high quality.
Model Development: This step involves developing the analytical model that will be used to analyze the data. This includes selecting the appropriate modeling techniques, building the model, and validating the model. The goal is to ensure that the model is accurate and reliable.
Model Deployment: This step involves deploying the model to a production environment. This includes integrating the model with existing systems, ensuring that it is scalable, and monitoring its performance. The goal is to ensure that the model is operational and can be used to make decisions.
Model Monitoring: This step involves monitoring the performance of the model in the production environment. This includes tracking key performance indicators (KPIs), identifying any issues, and making necessary adjustments. The goal is to ensure that the model continues to perform well over time.
Model Maintenance: This step involves maintaining the model over time. This includes updating the model as new data becomes available, retraining the model as needed, and ensuring that it continues to meet business requirements. The goal is to ensure that the model remains relevant and effective.
Model Retirement: This step involves retiring the model when it is no longer needed or when a new model has been developed. This includes archiving the model, documenting the retirement process, and ensuring that all stakeholders are informed. The goal is to ensure that the model is retired in a controlled manner and that any necessary knowledge transfer occurs.

User Interface Development Process

User Interface (UI) Design: This step involves designing the user interface for the data analytics solution. This includes creating wireframes, mockups, and prototypes to visualize the UI design. The goal is to ensure that the UI is user-friendly and meets the needs of the end-users.
UI Deployment: This step involves deploying the UI to a production environment. This includes integrating the UI with existing systems, ensuring that it is scalable, and monitoring its performance. The goal is to ensure that the UI is operational and can be used by end-users.
UI Monitoring: This step involves monitoring the performance of the UI in the production environment. This includes tracking key performance indicators (KPIs), identifying any issues, and making necessary adjustments. The goal is to ensure that the UI continues to perform well over time.
UI Maintenance: This step involves maintaining the UI over time. This includes updating the UI as new features are added, retraining the UI as needed, and ensuring that it continues to meet business requirements. The goal is to ensure that the UI remains relevant and effective.

Detailed Steps in the Advanced Analytics Development Process

When a project is initiated, the data analytics team will use a series of project templates to guide the development process.

Code Repository

A project should have a code repository that contains the code and documentation for the project. The code repository should be organized in a way that makes it easy to find and understand the code. The code repository should also include documentation that explains how to use the code and how to run the project.

The code repository should be hosted in Github or Azure DevOps. See the Code Repository document for more information on how to set up a code repository.

ARC Data Analytics Handbook