Blog

Developing A Model-Based Systems Engineering Capability

Posted on: November 8th, 2016 by Lou Wheatcraft No Comments

 That Meets the Needs of Your Organization

This blog, and a previous blog series entitled “Model-Based Systems Engineering – A Data-Centric Perspective” are outgrowths from  discussions concerning Model-Based Systems Engineering (MBSE) and requirements engineering that occurred within the Requirements Working Group (RWG) sessions during International Council of Systems Engineering (INCOSE) International Workshop (IW) 2016 in Torrance, CA.

The intended target audience of this blog article are project and product managers and systems engineers who are stakeholders in activities that are defined by the systems engineering (SE) discipline and are thinking about, or are in the process of, implementing Model-Based Systems Engineering (MBSE) within their organization.

Many who are assessing whether or not they should implement MBSE are confused as to exactly what MBSE is and why they should implement an MBSE approach to their system development process.  Moreover, many wonder how to successfully implement the intent of MBSE within their organization irrespective of the size and complexity of the system under development and the size and culture of the organization developing the system.

The goals of this blog are two-fold:

  1. provide organizations with a better understanding of  why they need to implement MBSE within their organization and
  2. provide a roadmap that can be used to define and successfully implement, incrementally, a MBSE capability that is tailored to the unique needs of the organization, whether small, medium, or large enterprises.

This blog series on developing an MBSE capability is divided into three parts:

  1. In this first part of the series, I address the question “Why implement MBSE?”, listing benefits for doing so as well as discussing how MBSE enables an organization to use measures to better manage their system development efforts.
  2. In the second part of this series, I introduce the concept of MBSE capability levels (MCLs) and discuss the need to choose an MBSE toolset appropriate for the MCL chosen.
  3. In the third part of this series, I provide guidance and a roadmap an organization can follow to implement the chosen MCL.

Why Implement MBSE?

MBSE compliments the SE development lifecycle processes by enabling the opportunity for overall better quality, lower cost, and lower risk by providing the following benefits: (Note: This list of benefits is derived from a similar list documented in NASA’s “Expanded Guidance for NASA Systems Engineering”, Volume 2, Chapter 8.2.(NASA 2016))

  1. Meet the challenges associated with increasing complexity for current and future systems. (These challenges are listed in my previous blog: MBSE: Introduction.)
  2. Providing greater consistency of all products because any single piece of design information can be expressed authoritatively in a single place that can later be referred to by others for decisions, derivations, or formation of artifacts. 
(A more detailed discussion on SE artifacts can be found in my previous blog: MBSE: Introduction.)
  3. Providing better visibility into the salient characteristics of a system because multiple views can be created that succinctly address specific stakeholder concerns.
  4. Providing greater congruence and configuration management between documentation and reality:
SE artifacts can be generated automatically as differing views of the SE data set, lowering the effort to keep them up to date with the result that artifacts can always match the best available, current information.
  5. Establishing “ground truth”. Ground truth is the only true reality…. No matter what someone says or thinks, no matter what they “remember” or what perspective they have concerning what is being done or built or a decision that was made, if it isn’t in the model, it is’t the truth. (Requires all the artifacts to be maintained and kept current.)
  6. Facilitating the navigation, traceability, and interrogation of information. Managers and engineers can have access to the information more quickly and on an as-needed basis without going through manual distribution or search processes.
  7. Facilitating the management of the verification and validation activities. Data associated with verification can have higher quality, and provide greater insight in concerning the status of verification and validation activities. 
 This also allows data to be linked, and provides reports directly from the testing activity to the requirements if desired.
  8. Helping to identify and manage dependencies and address any inconsistencies across the system architecture prior to formal testing, lowering the costs for system verification and system validation.
  9. Reducing the costs associated with erroneous design and resulting rework because analysis of the MBSE artifacts can reveal a flaw as soon as it is created, enabling correction before downstream work is done, work that would be invalid, and costly and time consuming to correct if the upstream mistake were not corrected immediately.
  10. Providing identification, management, interoperability, and integration of information across business or organizational elements needed to support program budget and schedule goals. With the ability to metatag data, information, and artifacts, you can, for example, tie these things directly to your WBS.
  11. Ensuring that data needed by programs and projects (e.g., for milestones, reviews, mission operations, and anomalies or investigations, decisions, and outcomes) are identified and managed to provide traceability of the data used in decision-making.

Organizations need to develop a level of organizational MBSE capability that will enable them to realize the benefits listed above.  One size doesn’t fit all, so organizations need to access the MBSE capabilities that best fit their domain, product line (degree of complexity), and culture.

The level of MBSE capability an organization establishes needs to be tailored to the size and complexity of systems developed by the organization, whether small, medium, or large projects. To establish an appropriate SE capability, organizations need to address their people, processes, and tools.  People need the training, knowledge, and experience appropriate to the level of MBSE capability being implemented; processes need to be defined from the enterprise level to the project level that support the chosen level of MBSE capability, and appropriate tools and infrastructure need to be provided by the enterprise.

Unfortunately, “MBSE” often means different things to different people, with interpretation varying quite dramatically from those individuals involved in early lifecycle activities (such as conceptual design) to those involved in later activities such as detailed design and development.  Many are having a hard time trying to understand the hype about MBSE as it is currently being discussed and presented.  In our Requirements Working Group (RWG) discussions at the International Council of Systems Engineering (INCOSE) International Workshop 2016 (IW 2016), several INCOSE members wanted  to learn what MBSE was and how their organization could benefit from adopting MBSE.  They had attended several different IW sessions focused on MBSE, however the discussions were too technical and did not help them in their quest.  Many of the outcomes documented in this paper resulted from the RWG discussions that took place in the attempt to explain what MBSE really is to assist organizations to efficiently adopt an MBSE approach.  Without any upper-level guidance, adoption of MBSE can be overwhelming.

Measures – Using Data to Better Manage your SE Projects

A key advantage of adopting MBSE is being able to better manage, across all development lifecycle processes, the SE activities associated with our increasingly complex systems to monitor and access progress, identify issues, and make sure the system being developed will meet stakeholder needs and expectations.

As stated in the INCOSE “Systems Engineering Measurement Primer”, v2.0, (INCOSE 2010) using measures can “efficiently deliver information to systems engineering managers who use it for decision-making.”  Measurements help the project manager (PM) and systems engineer to:

  1. Monitor the progress and performance of SE and PM activities
  2. Communicate effectively throughout the organization
  3. Identify and correct problems early
  4. Make key tradeoffs
  5. Track specific project objectives
  6. Defend and justify decisions

Once the project/program has identified and defined the measures, the measures are managed closely throughout the SE development lifecycle processes. Due to the importance of a measure to project success, they are often classified as measures of performance (MOP) and measures of effectiveness (MOE) to be used by technical and programmatic leadership so that they may make informed decisions.

Measures and associated requirements that are high priority and considered critical to successful development and operations, are also referred to as Key Performance Parameters (KPPs), as a failure to meet a KPP requirement may put the project at risk of cost and/or schedule overruns, or at risk of performance shortfalls.  KPPs that are also “at risk” are often referred to as Technical Performance Measures (TPMs) by project management and closely monitored.  For systems based on new technologies that have not been proven in the actual operational environment, additional development and operational risk is added to the project.  Measures will help manage these risks.

KPPs that are tracked by management as TPMs will be monitored closely during implementation by comparing trends for the current actual achievement of the parameters with the values that were anticipated for the current time and projected for future dates.  Each major review needs to include a status of these measures.

MOPs, MOEs, KPPs, and TPMs can also be treated as Leading Indicators (LIs).  LIs are measures that provide information for evaluating effectiveness of how a specific activity is applied on a project (NASA 2016).  LIs provide information about impacts likely to affect the system performance objectives.  LIs may be an individual measure, or collection of measures, which can be used to plot trends (predict future state) before the actual performance is realized.  The goal is to provide insight into potential future states to allow management to take action before problems are realized.  Examples of LIs include measures associated with:

– Requirement trends: percent growth, To be Determined/To Be Resolved (TBD/TBR) closures, number of requirement changes, number of requirements approved, etc.

– Design trends: number of requirements implemented by design

– Interface trends: percent interface definitions documented and approved, TBD/TBR burn down (resolution/closure), number of interface requirements, number of interface requirement changes, etc.

– System Verification Trends: number of requirement verifications completed, number of deviations/waivers, etc.

– Review Trends: Review Item Descriptions (RIDs), Requests for Action (RFA), Action Items per review and burn down (closure), etc.

– Software Trends: number of requirements per build/release versus what was planned.

– Problem Report/Discrepancy Report Trends: number open/closed

A major source of measures are the attributes that can be defined as part of a requirement expression.   These attributes are discussed and defined in INCOSE-TP-2010-006-02, INCOSE “Guide to Writing Requirements”, Chapter 5 (INCOSE-TP-2010-006-02 2015) and my blog on requirement attributes. Using requirement attributes helps to better define measures to manage your project.  Given that requirements are the common threads that tie all systems engineering product development life-cycle processes together, having insight into these processes is necessary to manage the project effectively.  Using attributes, management is able to generate reports from metrics managed within your MBSE toolset like:

– How many, or what percentage, of requirements have been implemented in the design?

– For system verification, how may requirements have a verification approach defined?

– How many system verification activities have been successfully competed?  Have failed?

– What percentage of system validation activities have been completed?

The purpose of using measures is to provide management with metrics that need to be watched and tracked closely throughout the development lifecycle (and operations) to help ensure a successful program and mission.  The importance of identifying and managing using measures is emphasized by the number of documents that INCOSE has available in their store:

  1. Metrics Guidebook for Integrated Systems and Product Development (INCOSE‐TP‐1995‐002‐01).
  2. Systems Engineering Measurement Primer (INCOSE‐TP‐2010‐005‐02).
  3. Technical Measurement Guide (INCOSE-TP-2003-020-01).
  4. Systems Engineering Leading Indicators Guide (INCOSE-TP-2005-001-03).
  5. Project Managers Guide to SE Measurement for Project Success (INCOSE-TP-2015-001-01).

Using measures will help your organization address several of the challenges associated with increasing complexity of current and future systems:

– The need to capture, manage, and access all system and project/program management data and their associated interrelationships.

– The need to track progress, identify at risk activities, and take actions before these risks become problems that could impact cost, schedule, or being able to deliver a product that meets stakeholder needs in the operational environment.

– The need to transition from a “gut” decision-making culture to a data-driven decision culture that is more effective and appropriate for managing current and future complex systems.

– The need to integrate SE activities with the project/program management activities and resulting artifacts to better manage cost and schedule.

In the next part of this series, I introduce the concept of MBSE capability levels (MCLs) and the need to choose an MBSE toolset appropriate for the MCL chosen.

Comments to this blog are welcome.

If you have any other topics you would like addressed in our blog, feel free to let us know via our “Ask the Experts”  page and we will do our best to provide a timely response.

Tags: , , , , , , ,
Posted in Systems Engineering | Tagged , , , , , , , | Leave a comment

Leave a reply