What is the DIL?

Mitigate risk to production gateways by providing comprehensive integrated testing platform environments free of charge in a self-service, bi-directional modality.


The DIL manages an organization’s relationship as a profile where an organization will have an Administrative lead, someone responsible for the overall profile. The DIL will support an organizational profile maintaining several user accounts such as developers, analysts, testers, etc. An organizational profile will support the maintenance of several different gateways including UAT testing gateways, developer gateways, and configuring multiple endpoints and services for that organization. The organizational profile will also be used to manage communities of interest either being a member and joining a community or extending an organization’s contribution and engagement / contract with the DIL. All test history from various gateways can be rolled up at that profile level where they can see different things going on.

Testing End-Points

The DIL as a test harness is able to interact with externally routable IP address or DNS routable messaging, inbound and outbound bi-directional. In order for the DIL to initiate communication between a DIL managed Reference Implementation system (RI), the target System Under Test (SUT) endpoint must both be published and reachable by the DIL’s reference systems. These endpoints are web services with published WSDLs and published into the DIL’s Universal Description Discovery and Integration (UDDI) service. Within the NwHIN or eHealth Exchange, multiple specifications are currently supported. Therefore, when publishing a service endpoint, it is imperative that the version of service and other UDDI specific content is completely compliant to the specification.

Test Platform Lab

The DIL has been developed and supported by a team of message testing experts who have been participating in testing, enterprise class web service and message based exchanges for the past several years. It is the opinion of the DIL development team that many of the point and click or point-to-point testing tools COTS products currently available in the marketplace do not satisfy the testing requirements necessary to support the testing of interoperability. In order to test interoperability, and ensure 100% message delivery, the DIL development team architected and devised a test platform which serves to support test driven development, bi-directional with initiating and responding scenarios and test cases. The Test Lab utilizes a predictive testing model testing specifications and requirements as opposed to testing software by generating and capturing messages between initiator and responder type scenarios, the test harness or test platform lab has a complete message and response to a evaluate message compliance. This testing approach is unique to test platform labs such as the DIL.


The DIL offers an extensible audit logging and test validation subsystem. The DIL currently supports message archiving and retrieval for each test executed between SUT and the DIL. Unlike point and click COTS products that programmatically determine a pass or fail, the DIL is able to maintain a retrievable copy of the test and post execution evaluate the message against numerous criteria and represent a pass or fail outcome by leveraging its predictive testing model. This holds a tremendous value over some of the COTS products that do not maintain the test history between executions for comparative analysis.

Bi-directional Testing

The DIL was built from the ground-up to support bi-directional testing. As test cases are developed for either initiator or responder, a mirror test case is required to complete the full test case life cycle. To explain this, in order to develop an initiator scenario which initiates a request from a SUT to the DIL bi-directional, outbound, and initiator only, the DIL is required to respond to the initiated request. The system response is automatically generated based on the test being executed when the SUT is testing a responding scenario; the DIL will be responsible for the initiation message call. Alternatively, when the SUT is testing an initiator scenario, the DIL will be responsible for the responder message.

Testing Feedback and Log Analyzer

The Log Analyzer to come online in 2013 will advance the concept of predictive testing models. If the DIL understands how to initiate a request and understands what the response is supposed to be, based on the organization’s profile (HCID, AAID, etc.) and the test configuration and the test patient (patient demographics, etc.), the DIL will use its predictive testing models to figure out the appropriate value or each element of the message (whether initiator or responder).. During its evaluation, if it determines a message structure or content deviate from the specification, it knows not only that a deviation exists from what is supposed to be there, but it also knows what the value is supposed to be and why. Therefore, the log analyzer can highlight a specific area of the message which deviates from the specification and add / offer helpful hints to be compliant with the specification. The log analyzer can also be leveraged when new escaped defects are identified and coded into the DIL as new rules or ‘checkpoints’. Due to the DIL’s capability of retaining all prior test case executions, a participant of the DIL may elect to retroactively run these new test rules against prior executed test results; thus saving numerous hours retesting a system perhaps unnecessarily.

Executive Dashboard

The DIL seeks to provide communication mechanism between numerous levels of a project such as a tester to a test manager, a test manager to a project manager, project manager to a program manager, program manager to program executives. The DIL’s dashboard forms a visual report of the number of outstanding service sets, test scenarios, and test cases which are identified as needing to be executed as part of a software release. Progress and velocity can be quickly determined and monitored through regular publications of this dashboard. We encourage testers, developers, and managers to use these dashboards to proactively report their progress throughout the project lifecycle. The dashboard can also be used as a tool going into Test Readiness Review (TRR) or software release cycle. In order for test team acceptance of software from development, the DIL report shows all test cases successfully executed by the developers prior to handing over the software for full regression and system integrated testing. The DIL can be used to demonstrate that a product meets the required functionality. In the future, we see an alignment between published requirements and published test cases being a software vendor’s requirements to prove product compliance prior to being hired to develop. This in turn supports the Federal Government’s goal of doing more with less.


The DIL supports the concept of organizational profiles. The profiles can be used to create and support communities of interest. Communities of interest allow an organization to send out inclusive invites to other organizations that they intend to partner or exchange health information with. The Community approach lets us minimize the noise that an organization might receive against their testing endpoints which were publicly posted for all to see. By restricting / controlling the testing traffic to a particular organization the organization’s gateway will have significantly less traffic and analysis will be conducted in a shorter timeframe. With regards to teams, the DIL supports concept of an organizational profile. The organizational profile supports DYNAMIC intra-team members within an organization, and the community supports the community being seen within an organization.

Production Simulation

The DIL architecture supports as near to production ready systems as possible. Test driven development geared towards the support of interoperability within a simulated real-world environment is possible through the DIL. To support this, the DIL simulates 250 production type gateways to simulate health information exchange. This near production simulation provides a testing platform capable of mitigating or postponing significant interoperability and health information exchange challenges that otherwise could be delayed up to and into production.

Security Protocols

Numerous security requirements exist in order to protect the PII and PHI of health information being exchanged. The DIL closely emulates the security protocols seen by the production environments. The ability to test the exchange of health information in a realistic testing environment mitigates significant risk for unauthorized disclosure of confidential medical information.

Public Contribution of Test Analysis

The DIL seeks to foster collaboration between organizations testing in the DIL by evaluating their results against the specification and sharing their challenges with other organizations also testing in the DIL. This public collaboration of contributed test analysis will quickly identify specifications which have participant-identified implementation issues or requirements specifications which may require further attention.

Self Service

The DIL was developed to support a developer community working independently and having a need, at any hour of the day on any day of the week, to execute test cases bi-directionally between their reference system and the DIL. Self-service affords a developer working late on a Saturday night, for example, the ability to execute a test without relying on a colleague to participate in the test from a responding system. Self-service provides maximum flexibility for both developers and organizations that are building bi-directional, exchange-oriented systems between disparate systems up to the point of full integration testing. Supporting this level of testing ensures the organization’s system is adequately prepared and ready for system-to-system integration testing.

Support for Continuous Integration (CI)

Many development teams currently work in an environment supporting a continuous, integrated build service, thereby allowing development contributions to the development branch repository to occur on a daily basis. A continuous integration build server is able to dynamically determine if a build should be initiated on a nightly basis or on a frequency determined by the development team. If the build successfully compiles and deploys to a reference system, the DIL can be leveraged to execute a ‘lights out’ series of regression tests on a defined frequency, such as successfully building a new deployment by the CI each night. This integrated service provides the development and test team leads to determine the maturity and readiness of the software on a regular basis.

Support for Integration Testing

Beyond an initial set of compliance and conformance testing is a need to conduct extensive integration testing between disparate enterprise system stacks and other health information exchange systems and organizations. This is commonly referred to as integration testing. The DIL leverages its 250 operational gateways to test both integrated services beyond the domain level and into the network of networks and integrated ecosystem.