Forsvarsudvalget 2018-19 (1. samling)
FOU Alm.del Bilag 57
Offentligt
2014551_0001.png
FY18 DOD PROGRAMS
F-35 Joint Strike Fighter (JSF)
Executive Summary
Programmatics
• Block 3F Development
- The program completed System Design and Development
(SDD) flight testing in April 2018, but continued testing
new modernization increments of software to address open
deficiencies and improve performance.
- The program and stakeholders reviewed open deficiencies
between May and July, re-categorizing many of the 102
Category 1 deficiencies (as of May 2018) to Category 2,
leaving 13 open Category 1 deficiencies for entry into
IOT&E, which later became 15.
• IOT&E Readiness
- The program focused on preparations for IOT&E
readiness throughout FY18.
- The Defense Acquisition Executive certified the program
as ready for entry into formal IOT&E, provided eight
remaining readiness requirements are met prior to the start
of for-score events.
- DOT&E verified readiness and approved the F-35 IOT&E
Test Plan on December 3, 2018.
- The Joint Strike Fighter (JSF) Operational Test Team
(JOTT) began formal IOT&E open-air testing in
accordance with the plan on December 5, 2018.
• Continuous Capability Development and Delivery (C2D2)
- The JSF Program Office (JPO) and Lockheed Martin
began to transition the development effort from delivering
Block 3F capabilities in the SDD contract to a more rapid
development, testing, and fielding cycle for additional
capabilities in Block 4, and to address deficiencies carried
over from SDD.
- DOT&E considers the current C2D2 schedule to be high
risk due to the large amount of planned capabilities to be
delivered in 6-month increments.
Operational E ectiveness
• Operational Testing
- The JOTT began conducting pre-IOT&E early test events
for score in January 2018 with cold weather testing,
followed by additional testing starting in April, including
two-ship scenarios, deployments, and weapons testing.
• Mission Data Load (MDL) Development and Testing
- The U.S. Reprogramming Laboratory (USRL)
demonstrated the capability to create functioning MDLs
for Block 3F and earlier blocks during SDD; however,
it still lacks adequate equipment to be able to fully test
and optimize MDLs under stressing conditions to ensure
adequate performance against current and future threats.
- Significant additional investments, well beyond the
current upgrades to the signal generator channels and
reprogramming tools, are required now for the USRL to
support F-35 Block 4 C2D2 MDL development.
Operational Suitability
• Autonomic Logistics Information System (ALIS)
- The program completed fielding of ALIS 2.0.2.4 in early
CY18 and focused on testing the next iteration of the
software, version 3.0.1.
- Two additional versions of ALIS 3.0.1 software were
developed and tested – versions 3.0.1.1 and 3.0.1.2 – to
address deficiencies before delivery to fielded units.
• Cybersecurity Operational Testing
- During CY18, the JOTT assessed ALIS version 3.0, F-35
training systems, and the ALIS-to-shipboard network
interface onboard a nuclear powered aircraft carrier.
- Cybersecurity testing in 2018 showed that some of the
vulnerabilities identified during earlier testing periods still
had not been remedied.
- Limited cybersecurity testing of the air vehicle is planned
during IOT&E; more testing will be needed.
• Availability, Reliability, and Maintainability
- There was no improving trend in fleet aircraft availability
- Fleet-wide average availability is below program target
value of 60 percent and well below planned 80 percent
needed for efficient conduct of IOT&E.
- The trend in fleet availability has been flat over the past 3
years; the program’s reliability improvement initiatives are
still not translating into improved availability.
- Reliability and maintainability metrics defined in the JSF
Operational Requirements Document are not meeting
interim goals needed to reach requirements at maturity.
Live Fire Test and Evaluation (LFT&E)
• In FY18, Lockheed Martin completed the Vulnerability
Assessment Report and the Consolidated LFT&E Report.
These reports do not include results from Electromagnetic
Pulse (EMP) or gun lethality testing, which were not
completed by the end of FY18.
JSF
23
 FOU, Alm.del - 2018-19 (1. samling) - Bilag 57: Henvendelse af 9/2-19 fra Heine Strømdahl, Gentofte, vedrørende F-35 problemer og den seneste rapport fra DOT og E
2014551_0002.png
FY18 DOD PROGRAMS
• DOT&E is reviewing the F-35 vulnerability reports and
completing its own evaluation, which will be documented
in the combined IOT&E and LFT&E report to be published
prior to the Full-Rate Production decision, anticipated in
FY20.
• The JPO evaluated the chemical and biological agent
protection and decontamination systems during dedicated
full-up system-level testing. However, the test plan to
assess the chemical and biological decontamination of
pilot protective equipment is not adequate because the JPO
does not plan to test the decontamination process for either
the Generation (Gen) III or Gen III Lite Helmet-Mounted
Display System (HMDS).
• Air-to-ground flight lethality tests of three 25-mm round
variants against armored and technical vehicles, small
boats, and plywood mannequins were conducted at the
Naval Air Warfare Center Weapons Division (NAWCWD)
at NAWS China Lake, California, from August through
December 2017. The rounds tested were the Projectile Gun
Unit (PGU)-32/U Semi-Armor-Piercing High-Explosive
Incendiary round, PGU-47/U Armor-Piercing
High-Explosive Incendiary with Tracer round, and
PGU-48/B Frangible Armor-Piercing round. The target
damage results are classified.
System
• The F-35 JSF program is a tri-Service, multinational,
single-seat, single-engine family of strike aircraft consisting
of three variants:
- F-35A Conventional Take-Off and Landing
- F-35B Short Take-Off/Vertical-Landing
- F-35C Aircraft Carrier Variant
• The F-35 is designed to survive in an advanced threat
environment (year 2015 and beyond). It is also designed
to have improved lethality in this environment compared to
legacy multi-role aircraft.
• Using an active electronically scanned array radar and
other sensors, the F-35 with Block 3F or later software is
intended to employ precision-guided weapons (e.g., GBU-12
Laser-Guided Bomb, GBU-31/32 JDAM, GBU-39 Small
Diameter Bomb, Navy Joint Stand-Off Weapon version
C1) and air-to-air missiles (e.g., AIM-120C Advanced
Medium-Range Air-to-Air Missile (AMRAAM), AIM-9X
infrared-guided, air-to-air missile) and a 25 mm Gun
Automatic Unit (GAU)-22/A cannon.
• The SDD program was designed to provide mission
capability in three increments:
- Block 1 (initial training; two increments were fielded:
Block 1A and Block 1B)
- Block 2 (advanced training in Block 2A and limited
combat capability with Block 2B)
- Block 3 (limited combat capability in Block 3i and full
SDD warfighting capability in Block 3F)
• The F-35 is under development by a partnership of
countries: the United States, United Kingdom (UK), Italy,
the Netherlands, Turkey, Canada, Australia, Denmark, and
Norway.
Mission
• The Combatant Commander will employ units equipped
with F-35 aircraft in joint operations to conduct a variety of
missions during day or night, in all weather conditions, and
in heavily defended areas.
• The F-35 will be used to attack fixed and mobile land targets,
surface units at sea, and air threats, including advanced
aircraft and cruise missiles.
Major Contractor
Lockheed Martin, Aeronautics Company – Fort Worth, Texas
Programmatics
Block 3F Developmental Testing
•
Activity
- The program completed SDD developmental flight testing
on April 11, 2018, after nearly 10 years of flight testing.
- At the completion of Block 3F developmental flight testing
in April, the program had 941 open deficiencies – either
in work or under investigation. These included 102
Category 1 deficiencies and 839 Category 2 deficiencies.
- The Integrated Test Force (ITF) published their report on
Block 3F testing in March 2018. The report documented
numerous open deficiencies across the air system in the
final version of Block 3F software, 18 of which were
designated Category 1. The ITF recommended that the
deficiencies be corrected, although the system could
proceed into IOT&E.
- As of October 17, 2018, the JPO had collected data and
verified performance to close out 475 of 536 (89 percent)
contract specifications paragraphs. Additionally, 3,363
of 3,452 (97 percent) success criteria derived from the
contract specifications had been completed.
- The program continued to address documented
deficiencies in the Block 3F software by developing
and flight testing additional software versions, under
the nomenclature of Block 30RXX, as part of planned
modernization. Throughout CY18, the program developed
and tested numerous iterations, including versions 30R00,
30R01, and 30R02, and associated “Quick Reaction
Cycle” versions (e.g., 30R01.02) to correct deficiencies
and improve performance.
24
JSF
 FOU, Alm.del - 2018-19 (1. samling) - Bilag 57: Henvendelse af 9/2-19 fra Heine Strømdahl, Gentofte, vedrørende F-35 problemer og den seneste rapport fra DOT og E
2014551_0003.png
FY18 DOD PROGRAMS
- The test centers at Edwards AFB, California, and Naval
Air Station (NAS) Patuxent River, Maryland, made plans
to transition test aircraft from Block 3F SDD to follow-on
modernization. The status and configuration of the 18
developmental test aircraft used for SDD testing as of the
end of September 2018 are as follows: 3 were retired, 2
were in storage, 5 were available for flight sciences testing,
5 were continuing missions systems testing, and 3 were
returned to the Marine Corps and Navy as operational test
aircraft.
- The program and stakeholders reviewed open deficiency
reports between May and July, re-categorizing many
of the 102 Category 1 deficiencies (as of May 2018) to
Category 2, leaving 13 open Category 1 deficiencies for
entry into IOT&E, which later became 15.
•
Assessment
- Although the program completed SDD flight testing in
April, the test centers continued to work on Block 3F
technical debt by addressing known deficiencies. The
extent that the open deficiencies will affect combat
capability will be assessed during IOT&E.
Static Structural and Durability Testing
•
Activity
- The F-35A full scale durability test article (AJ-1)
completed the third lifetime of testing (one lifetime is
8,000 equivalent flight hours (EFH) on October 17, 2017.
The test article was delivered to an inspection facility
in June 2018, and is currently undergoing disassembly,
inspections, and analysis.
- The program suspended testing of the F-35B ground test
article (BH-1) after completing the second lifetime of
testing in February 2017. Due to the significant amount
of modifications and repairs to bulkheads and other
structures, the program declared the F-35B ground test
article no longer representative of the wing-carry-through
structure in production aircraft, deemed it inadequate
for further testing, and canceled the testing of the third
lifetime with BH-1. The program secured funding
to procure another ground test article, which will be
production-representative of Lot 9 and later F-35B aircraft
built with a re-designed wing-carry-through structure, but
to date does not have the procurement of the test article on
contract. The program has not completed durability testing
of the aircraft with the new wing-carry-through structure
to date.
- The F-35C durability test article (CJ-1) began third
lifetime testing on April 4, 2017, and reached 18,792
EFH on April 12, 2018. Testing was stopped at that time
following the discovery of more cracking in the Fuselage
Station (FS) 518 Fairing Support Frame (cracking had
been discovered at the end of the second lifetime),
requiring repair before additional testing could proceed.
After making an estimate for the cost and time to repair
or replace the FS 518 Fairing Support Frame, coupled
with the need to manage other structural parts that had
existing damage (fuel floor segment, FS 450 bulkhead,
FS 496 bulkhead, FS 556 bulkhead, and front spar repair)
via scheduled inspections, the program determined that
the third lifetime testing should be discontinued. The test
article was removed from the test fixture in August 2018
and prepped for shipment to the tear down and inspection
facility in September. Although the program planned
for a third lifetime of testing to accumulate data for life
extension, if needed, the program currently has no plans to
procure another F-35C ground test article.
•
Assessment
- For all variants, this testing has led to discoveries requiring
repairs and modifications to production designs, some as
late as Lot 12 aircraft, and retrofits to fielded aircraft.
- Based on durability testing, the service life of
early-production F-35B aircraft is well under the expected
service life of 8,000 flight hours, and may be as low as
2,100 flight hours. Fleet F-35B aircraft are expected to
start reaching their service life limit in CY26, based on
design usage. The JPO will continue to use Individual
Aircraft Tracking (IAT) of actual usage to help the
Services project changes in timing for required repairs and
modifications, and aid in Fleet Life Management.
- For the F-35C, expected service life will be determined
from the durability and damage tolerance analysis
following tear down.
IOT&E Readiness
•
Activity
- The JPO, Lockheed Martin, and JOTT continued to make
preparations for entry into formal IOT&E.
- On August 24, 2018, DOT&E provided guidance in a
memorandum to the test agencies on detailed requirements
for formal entry into IOT&E. Specifically, to add clarity
to the formal entrance criteria, the following items were
listed as requirements for formal start:
▪ F-35 software version Block 30R02 with Level 4 (fully
validated and verified) mission data files (MDF)
▪ ALIS software version 3.0
▪ Air-to-Air Range Infrastructure (AARI) system with
corrections planned for Block 30R02 software.
- On October 2, 2018, the Defense Acquisition Executive
certified the program as ready for entry into formal IOT&E
provided eight remaining readiness requirements are met
prior to the start of for-score events:
▪ A fully validated and verified mission data file for the
Block 30R02.03 software
▪ U.S. Services airworthiness authorities provide flight
clearances for each variant with the Block 30R02.03
software
▪ The program provides flight series data and joint
technical data updated for the Block 30R02.03 software
▪ Full partner participation is authorized for the applicable
portions of the IOT&E mission sets
▪ The last OT aircraft undergoing depot modifications –
BF-18 – is delivered to Edwards AFB
JSF
25
 FOU, Alm.del - 2018-19 (1. samling) - Bilag 57: Henvendelse af 9/2-19 fra Heine Strømdahl, Gentofte, vedrørende F-35 problemer og den seneste rapport fra DOT og E
2014551_0004.png
FY18 DOD PROGRAMS
▪ Accreditation of necessary models for use in IOT&E are
completed or on track for use
▪ All unit-level modifications to the OT aircraft are
complete, except those specifically waived or deferred
by DOT&E
▪ AARI has been installed on aircraft BF-17, BF-18, and
CF-8 (the last three U.S. OT aircraft to complete depot
modifications).
- DOT&E approved the IOT&E test plan on December 3,
2018, after verifying that the remaining readiness actions
listed above had been met.
•
Assessment
- Two additional factors caused readiness for the formal
start of IOT&E to slip into early December. A Category 1
deficiency associated with blanking of the cockpit displays
was discovered in Block 30R02.03 software, causing an
additional software patch called 30R02.04 to be developed
and tested prior to start of formal IOT&E. Additionally,
a fleet-wide grounding in October 2018 to inspect and
replace fuel pump tubes in a number of the OT aircraft
added to the delay in readiness to start.
Continuous Capability Development and Delivery (C2D2)
•
Activity
- The JPO and Lockheed Martin began to transition the
development effort from delivering Block 3F capabilities
in the SDD contract to a more rapid development, testing,
and fielding cycle for additional capabilities in Block 4 and
to address deficiencies carried over from SDD.
- The program’s plans for the Block 4 modernization are
included in an updated F-35 acquisition strategy that was
approved on October 16, 2018.
▪ These plans include lean test designs and agile
development tenets.
▪ The developmental test effort will be government-led
compared to the contractor-led approach used for SDD.
▪ The program plans to leverage a greater dependence on
modeling and simulation than was used during SDD.
- The program developed and began staffing a draft Test
and Evaluation Master Plan (TEMP) to support Block 4
development activities.
•
Assessment
- The current C2D2 schedule is high risk with the planned
content of capabilities to be made available for delivery in
6-month increments.
- Many of the lessons learned from SDD involving the
amount of testing that can be done in laboratories and
simulations, vice flight testing, could be applied to C2D2
planning.
- The program needs to ensure adequate funding is available
to support a robust laboratory and simulation environment
and develop adequate verification, validation, and
accreditation plans.
- Sustaining multiple configurations of fielded aircraft (i.e.,
Block 2B, Block 3F, and the new electronic warfare (EW)
system in Lot 11 and later aircraft) while managing a
developmental test fleet with updated hardware to support
the production of new lot aircraft will be a challenge for
the JPO.
- The cost of software sustainment for multiple
configurations of aircraft needs to be adequately assessed.
- The planned 6-month software release cycle does not align
with the timelines of other increments of capability needed
to support the entire JSF system (i.e., ALIS, mission data,
training simulators, aircraft modifications). Other modern
fighters (e.g., F/A-18, F-22) have historically taken much
longer than 6 months – 2 and 3 years, respectively – to
field new increments of capability. A more realistic
C2D2 schedule with achievable content releases that
includes adequate test infrastructure (labs, aircraft, and
time) and modifications while aligning the other fielding
requirements is necessary.
- F-35 modernization is on OT&E oversight. DOT&E will
review the content of each Block 4 increment and, if the
increment contains significant new capabilities or new
hardware, it will require a tailored formal OT&E. DOT&E
routinely conducts “agile” OT for other programs, so each
F-35 OT&E will be tailored to be as efficient as possible
while maintaining test adequacy by leveraging integrated
testing with developmental testing (DT) and focusing on
evaluating the new capabilities and affected mission areas.
Operational E ectiveness
Operational Testing
•
Activity
- DOT&E, in coordination with the JPO and the JOTT,
approved execution of select for-score pre-IOT&E test
activities, prior to satisfying all 47 TEMP readiness criteria
for IOT&E, when the applicable readiness criteria were
met and the testing could be adequately completed.
▪ Pre-IOT&E Increment 1: On January 18, 2018, DOT&E
approved the JOTT to conduct planned cold weather
testing that occurred from January 18 to February 2,
2018, at Eielson AFB, Alaska. The operational test
squadrons deployed six F-35 aircraft, two of each
variant, from Edwards AFB, California. The purpose
of this for-score testing was to evaluate the suitability
of the F-35 air system and evaluate alert launch
timelines in the extreme cold weather environment. The
deployment was one of six required by the F-35 IOT&E
test design.
▪ Pre-IOT&E Increment 2: Following approval from
DOT&E on March 30, 2018, the JOTT began for-score
testing of limited two-ship mission scenarios with
Block 3F (30R00) software and Level 2 MDFs. The
scenarios included Close Air Support, Reconnaissance,
Forward Air Controller-Airborne, Strike Coordination
and Armed Reconnaissance, and Combat Search and
Rescue, along with ship deployments and weapons
delivery events. Some missions were re-flown by the
26
JSF
 FOU, Alm.del - 2018-19 (1. samling) - Bilag 57: Henvendelse af 9/2-19 fra Heine Strømdahl, Gentofte, vedrørende F-35 problemer og den seneste rapport fra DOT og E
2014551_0005.png
FY18 DOD PROGRAMS
A-10 as part of the planned F-35A and A-10 comparison
testing.
▪ The JOTT and the F-35A operational test squadrons
deployed four F-35A OT aircraft from June 4 – 29,
2018, to Eglin AFB, Florida, to conduct Pre-IOT&E
air-to-air missile Weapons Demonstration Events over
the Gulf Coast test ranges. During the deployment,
the test team completed six AIM-120 and six AIM-9X
missile events, some with multiple shots, and all in
accordance with the approved plan. In limited cases,
DOT&E approved modifications to the mission profile
when warranted.
▪ The JOTT, in coordination with VFA-125, the Navy’s
west coast F-35C Fleet Replacement Squadron,
deployed six aircraft aboard the USS
Abraham Lincoln
from August 18 – 31, 2018, to conduct shipboard
operations and evaluate F-35C sortie generation rate
(SGR) capabilities, per the IOT&E test plan.
▪ The test included participation of aircraft from Carrier
Air Wing Seven, which provided an operationally
representative flight deck environment. This was the
first time the F-35C was integrated with the rest of
a carrier air wing as it would during an operational
deployment.
▪ The Navy approved the use of the F-35 Integrated
Power Package (IPP) in the hangar bay for maintenance
purposes, on an interim basis, just prior to the SGR
testing onboard CVN 72. This approval will enable
more efficient maintenance during deployments,
increasing the options for providing electrical power
and cooling air to aircraft undergoing maintenance.
Squadrons will use temperature sensing devices to
ensure that the IPP exhaust, which vents upwards on
the F-35C, does not damage hangar bay overhead
equipment, cabling, and structure while in use.
▪ The Navy finalized a design for the Closed Bay Fire
Fighting Tool (CBFFT), and produced several examples
to provision CVN 72’s crash and fire personnel prior
to the SGR testing. The CBFFT will allow emergency
responders to cut through the exterior of an F-35 aircraft
carrying live internal ordnance and plug a water hose
into the hole to provide ordnance cooling during a fire
on the flight deck.
▪ The JOTT and the F-35A operational test squadron
deployed four F-35A OT aircraft to Volk Field
Air National Guard Base, Wisconsin, to evaluate
sortie generation rate surge operations from
September 10 – 16, 2018. Although the test plan called
for six aircraft to deploy, two remained at Edwards AFB
due to maintenance problems.
Assessment
- DOT&E will report the results of the pre-IOT&E test
events following IOT&E.
Gun Testing
•
Activity
- All three F-35 variants have the GAU-22/A cannon. The
F-35A gun is internal; the F-35B and F-35C each use
an external gun pod. Differences in the outer mold-line
fairing mounting make the gun pods unique to a specific
variant (i.e., an F-35B gun pod cannot be mounted on an
F-35C aircraft).
- Through July 2018, 19 air-to-ground strafing missions
had been completed to assess gun accuracy on the F-35A.
Eighteen missions were flown with AF-31 and one mission
with AF-80. Over 3,400 rounds were fired using a cross
section of rounds, including PGU-23, PGU-47, and
PGU-48.
- Through July 2018, 13 air-to-ground strafing missions had
been completed using the missionized gun pod; one on
BF-15, one on BF-16, six on BF-17, and five on CF-08.
Overall, 2,695 rounds were fired using PGU-23 and
PGU-32 rounds, including some for assessing accuracy
compliance.
- Operational test pilots conducted live firings of the gun
against airborne targets, including drones and towed
banners, throughout CY18. These firings were often in
combination with other weapon demonstration events,
such as air-to-air missile employment events.
•
Assessment
- Based on F-35A gun testing through September 2018,
DOT&E currently considers the accuracy of the gun, as
installed in the F-35A, to be unacceptable.
- F-35A gun accuracy during SDD failed to meet the
contract specification. Although software corrections were
made to the F-35 mission systems software to improve
the stability of gun aiming cues, no software or hardware
corrections have yet been implemented to correct the gun
accuracy errors.
- Investigations into the gun mounts of the F-35A revealed
misalignments that result in muzzle alignment errors. As a
result, the true alignment of each F-35A gun is not known,
so the program is considering options for re-boresighting
and correcting gun alignments.
- During air-to-air gun testing, F-35A operational test pilots
received intermittent “unsafe gun” cockpit alerts while
attempting gun attacks. These alerts occurred with two
different aircraft; the root cause is under investigation.
- F-35B and F-35C air-to-ground accuracy results to date
with the gun pod have been consistent and meet the
contract specifications. They do not show the accuracy
errors of the internal gun on the F-35A.
Mission Data Load (MDL) Development and Testing
•
Activity
- F-35 effectiveness relies on the MDL, which is a
compilation of the mission data files (MDF) needed for
•
JSF
27
 FOU, Alm.del - 2018-19 (1. samling) - Bilag 57: Henvendelse af 9/2-19 fra Heine Strømdahl, Gentofte, vedrørende F-35 problemer og den seneste rapport fra DOT og E
2014551_0006.png
FY18 DOD PROGRAMS
operation of the sensors and other mission systems. The
MDL works in conjunction with the avionics software and
hardware to drive sensor search behaviors to provide target
identification parameters. This enables the F-35 avionics to
identify, correlate, and respond to sensor detections, such
as threat and friendly radar signals.
▪ The contractor produces an initial set of MDLs for each
software version to support DT during SDD.
▪ The USRL at Eglin AFB, Florida, creates, tests, and
verifies operational MDLs – one for OT and training,
plus one for each potential major geographic area of
operation, called an area of responsibility (AOR).
OT aircraft and fielded aircraft use the applicable
USRL-generated MDLs for each AOR.
- The testing of the USRL MDLs is an operational test
activity, as arranged by the JPO after the program
restructure that occurred in 2010, and consists of
laboratory and flight testing on OT aircraft.
Assessment
- Because MDLs are software components essential to
F-35 mission capability, the Department must have a
reprogramming lab that is capable of rapidly creating,
testing, and optimizing MDLs, as well as verifying their
functionality under stressing conditions representative of
real-world scenarios.
▪ The USRL demonstrated the capability to create
functioning MDLs for Block 3F and earlier blocks
during SDD. However, it still lacks adequate equipment
to be able to test and optimize MDLs under conditions
stressing enough to ensure adequate performance against
current and future threats in combat.
▪ The lab lacks a sufficient number of high-fidelity radio
frequency signal generator channels, which are used to
stimulate the F-35 EW system and functions of the radar,
with simulated threat radar signals. This situation is
improving as of the writing of this report, but additional
improvements, above and beyond those currently
planned, will be required.
▪ By late 2019, both USRL mission data test lines will
have been upgraded from three to eight high-fidelity
channels. Eight high-fidelity channels per line
represents a substantial improvement, but is still far
short of the 16-20 recommended in the JPO’s own 2014
gap analysis.
▪ Even when this upgrade is complete, the USRL will
still not have enough signal generators to simulate a
realistic, dense threat laydown with multiple modern
surface-to-air missile threats and the supporting
air defense system radars that make up the signal
background in the laydown.
- The reprogramming lab must also be able to rapidly
modify existing MDLs when intelligence data changes.
▪ The mission data reprogramming hardware and software
tools used by the USRL during SDD were cumbersome,
requiring several months for the USRL to create, test,
optimize, and verify a new MDL for each AOR. For this
reason, effective rapid reprogramming capability was
not demonstrated during SDD.
▪ This situation recently improved with the delivery of
a new Mission Data File Generation (MDFG) tool set
from the contractor. How much improvement these
tools will bring to MDL development timelines is yet
to be determined, but initial indications are that the
improvements will be significant.
- Significant additional investments, well beyond the current
upgrades to the signal generator channels and MDFG
tools, are required now for the USRL to support F-35
Block 4 C2D2 MDL development.
▪ The C2D2 plan includes new avionics hardware.
Concurrency in development and production during
SDD resulted in multiple fielded F-35 configurations
that will continue to need to be supported indefinitely
(i.e., until a specific configuration is modified or retired),
after the development program enters the C2D2 phase.
During C2D2, the program will require the USRL, or
an additional reprogramming lab, to have the capability
to simultaneously create and test MDLs for different
avionics hardware and software configurations. These
different configurations include the fielded Technical
Refresh 2 processors for Block 3F, new EW equipment
in Lot 11 and later aircraft, an improved display
processor, new Technical Refresh 3 open-architecture
processors, and other avionics for later increments in
C2D2.
▪ In order to be on a timeline that is fully aligned with
the planned C2D2 capability development timeline, the
C2D2 hardware upgrades for the USRL should have
already been on contract. However, the requirements
for the C2D2 software integration lab have yet to be
fully defined. The JPO must expeditiously complete
the development of these requirements while ensuring
adequate lab infrastructure to meet the aggressive
development timelines of C2D2 and the operational
requirements of the Block 4 F-35.
- As part of IOT&E, the USRL will complete an Urgent
Reprogramming Exercise (URE). This test event will
evaluate the ability of the USRL, with its hardware and
software tools, to respond to an urgent request to modify
the mission data in response to a new threat or a change to
an existing threat.
▪ During a URE at the USRL in 2016, the total hours
recorded were double the Air Force standard for rapidly
reprogramming a mature system. The JOTT identified
several key process problems, including the lack of
necessary hardware, analysis tools that were not built
for operational use, and missing capabilities, such as the
ability to quickly determine ambiguities in the mission
data.
▪ The JPO is working to correct these problems in order
to bring the ability of the USRL to react to new threats
up to the identified standards routinely achieved on
legacy aircraft. A new Ambiguity Analysis Tool (AAT),
•
28
JSF
 FOU, Alm.del - 2018-19 (1. samling) - Bilag 57: Henvendelse af 9/2-19 fra Heine Strømdahl, Gentofte, vedrørende F-35 problemer og den seneste rapport fra DOT og E
2014551_0007.png
FY18 DOD PROGRAMS
originally developed to meet requirements set forth
for the Australia-Canada-UK Reprogramming Lab
(ACURL), was delivered to both the ACURL and the
USRL. The initial version of the AAT has provided
improvements in identifying and correcting mission
data ambiguities. Enhancements to the AAT now in
work promise to significantly speed up the mission data
development process.
- In addition to resolving the laboratory deficiencies above,
the program will need to properly sustain the USRL to
ensure a high state of readiness, particularly if the Services
have an urgent reprogramming requirement, which could
happen at any time for the fielded aircraft. To meet these
tasks, the USRL will also need to maintain all necessary
equipment in a functioning status with a high rate of
availability, which will require a sufficient number of
prime contractor Field Service Engineers to assist in
maintenance and operation of the lab equipment, and
adequate training for laboratory personnel. In addition, the
USRL requires adequate technical data for lab equipment
and enough spare parts and/or supply priority to quickly
repair key components.
Joint Simulation Environment (JSE)
•
Activity
- The JSE is a man-in-the-loop, F-35 software-in-the-loop
mission simulator intended to conduct IOT&E scenarios
with modern threat types and densities that are not
able to be replicated in open air. Originally slated to
be operational by the end of 2017, first use of a fully
functional simulator is now planned for the beginning
of 2019 with accreditation later in 2019, near the end of
planned IOT&E trials.
- The JSE’s physical facilities (cockpits, visuals, and
buildings) and synthetic environment (terrain, threat,
and target models) are nearing completion and security
accreditation. Integration of the F-35 and its weapons is
planned for 1QFY19. The JSE verification and validation
process has made progress, but the bulk of validation
testing still remains for the first half of FY19.
•
Assessment
- The government-led JSE team made good progress this
year in getting the hardware developed and installed,
which will likely meet requirements for IOT&E.
- The planned schedules for JSE software development
and accreditation support IOT&E, but there is some
risk to software development (particularly F-35 model
integration), which also affects verification and validation.
Without the JSE, the IOT&E will be unable to adequately
assess the F-35 against dense and modern threats
that are not available for open-air testing, resulting in
operational risk. Once the JSE completes development
and accreditation, it should be an invaluable resource for
follow-on F-35 testing and possibly for testing of other
platforms.
Radar Signal Emulators (RSE)
•
Activity
- The Nevada Test and Training Range (NTTR) began
accepting Radar Signal Emulators in late CY16 to support
the DOT&E-initiated Electronic Warfare Infrastructure
Improvement Program (EWIIP). As of October 10, 2018,
9 of 16 emulators had been accepted on the NTTR and had
been used to conduct integration testing with the F-35 and
other range test assets.
- The RSEs will be used to provide operationally realistic
threat laydowns for use in F-35 IOT&E.
•
Assessment
- All 16 RSEs should complete acceptance testing and
integration by the end of CY18 and will be used to emulate
threats during IOT&E.
- More detail on the background, development, and fielding
of EWIIP can be found in the T&E Resources section of
this report.
Operational Suitability
Autonomic Logistics Information System (ALIS)
•
Activity
- The program completed fielding of ALIS 2.0.2.4 in early
2018. Feedback from operational users included:
▪ The Deployment Planning Tool did not work well or
significantly improve the ease of deploying F-35 units.
▪ Life Limited Parts Management, which includes
propulsion data integration and Production Aircraft
Inspection Requirements (PAIRs), requires a great
deal of time with manual workarounds by maintenance
personnel.
- The program rolled the capabilities planned for release in
ALIS 2.0.2.5 into the next block of software – ALIS 3.0.1.
ALIS 2.0.2.5 was intended to address deficiencies and
usability problems, upgrade the browser to Internet
Explorer 11, and include a filtering function to decrease
false alarms in the Prognostic Health Management (PHM)
System, referred to as Advanced Filter and Correlate
(AFC).
- The program focused on testing in preparation for fielding
ALIS software version 3.0.1 throughout CY18. This
version of ALIS software includes the following new
major capabilities:
▪ Support for lightning protection.
▪ Low Observable Health Assessment System (LOHAS)
improvements.
▪ Security enhancements.
▪ The first increment of the new Training Management
System for tracking maintainer qualifications.
▪ Improvements to address technical debt and corrections
to existing deficiencies.
JSF
29
 FOU, Alm.del - 2018-19 (1. samling) - Bilag 57: Henvendelse af 9/2-19 fra Heine Strømdahl, Gentofte, vedrørende F-35 problemer og den seneste rapport fra DOT og E
2014551_0008.png
FY18 DOD PROGRAMS
- The program conducted initial testing of ALIS 3.0.1
with field data between November 28, 2017, and
January 7, 2018.
▪ Testing with developmental test aircraft occurred at
the Air Force Test Center at Edwards AFB and NAS
Patuxent River.
▪ The Operationally Representative Environment (ORE)
at Edwards AFB was also used, which consists of
production-representative ALIS hardware in a closed
network and is designed for testing ALIS software
using data downloaded from OT aircraft. The ORE also
allows testing of ALIS propulsion capabilities as ALIS
cannot support SDD propulsion systems.
▪ Because of limitations associated with the hardware
versions of the ALIS equipment used to support the
SDD aircraft and the ORE, the program could not
conduct fully operationally representative testing of new
ALIS software versions in either venue.
- The initial report issued jointly by the test centers at
Edwards AFB and NAS Patuxent River recommended
that ALIS 3.0.1 continue development and testing before
fielding.
- After making several fixes, the program completed testing
of ALIS 3.0.1.1 with field data at the same venues between
April 3 and May 31, 2018, and recommended fielding of
this release. Findings included:
▪ Updated software corrected the erroneous recording
of air vehicle flight hours to components installed on
a different air vehicle, a deficiency identified during
ALIS 3.0.1 testing.
▪ Problems with existing ALIS 2.0.2.4 capabilities noted
in ALIS 3.0.1 testing were largely resolved.
▪ PHM performance improved as ALIS 3.0.1.1 eliminated
intermittent failures of PHM to auto-populate and
display data during debrief.
▪ AFC reduced non-actionable Health Reporting Codes
(HRC) and maintainer workload.
▪ Supply chain management data processing, data
accessibility of Electronic Equipment Logbooks (EELs),
which contain a virtual record of data for a specific
part, and Anomaly Fault Resolution System reliability
improved.
▪ Significant deficiencies in supporting aircraft
parts records remained, including long-standing
enterprise-wide problems with data quality.
▪ Documenting maintenance tasks in ALIS frequently
takes more time than completing the maintenance action.
▪ The lack of accurate and complete data in ALIS
continued to drive many workarounds.
▪ Deficiencies in the Deployment Planning Tool and in air
vehicle data transfer functionality were not resolved in
ALIS 3.0.1.1. Both require a high level of contractor
support with frequent work stoppages, creating a heavy
burden on support personnel time.
- The program completed verification testing of
ALIS 3.0.1.1 at Nellis AFB, Nevada, to evaluate some
capabilities, including LOHAS enhancements and
lightning protection, which the program could not fully
evaluate during prior testing. Following completion of
this verification period, the program approved the release
of ALIS 3.0.1.1 to operational test at Edwards AFB,
which took place in August 2018. Concurrently, the
program continued implementing fixes to ALIS 3.0.1.1
for the next software release, ALIS 3.0.1.2. The program
conducted initial testing of ALIS 3.0.1.2 on SDD aircraft
and at the ORE between June 9 and September 20,
2018, using five engineering releases. Initial testing was
followed by verification testing at Nellis AFB beginning
September 15, 2018. ALIS 3.0.1.2 does not deliver any
new capabilities, focusing instead on delivering fixes to
existing deficiencies. These fixes include:
▪ Improvements within ALIS reporting of the inert gas
state of the aircraft fuel system for lightning protection.
▪ A propulsion data processing anomaly introduced in
ALIS 3.0.1.1 was corrected.
▪ A deficiency introduced in ALIS 3.0.1.1 that caused
some damage tracings to not translate properly into
LOHAS, resulting in significant inaccuracies in LOHAS
status beyond the scope of actual damage, was corrected.
- The program installed ALIS 3.0.1.2 at the operational test
sites at Edwards AFB beginning on September 25, 2018;
it is expected to be the fielded version of ALIS that is
currently being used during formal IOT&E.
Assessment
- ALIS is designed to bring efficiency to maintenance and
flight operations, but it does not yet perform as intended.
User feedback on ALIS deficiencies, some of which can
have a significant effect on aircraft availability and sortie
generation, fall into three major categories:
▪ Users must employ numerous workarounds due to
data and functionality deficiencies. Most capabilities
function as intended only with a high level of manual
effort by ALIS administrators and maintenance
personnel. Manual workarounds are often needed to
complete tasks designed to be automated. Configuration
management of ALIS software and data products
remains complex and time-consuming.
▪ Users must deal with pervasive problems with data
integrity and completeness on a daily basis. Maintainers
frequently have to manually enter missing or incorrect
EEL data, which accompany spare parts, so they can be
accepted and tracked by an ALIS Standard Operating
Unit (SOU) at the squadron and installed on an aircraft.
Fixing data in complex EELs, which represent an
assembly such as ejection seats, requires a great deal
of time from ALIS administrators. EELs problems
have many sources, including vendors who have not
complied with guidance on creating EELs; a lack of
standardization among suppliers, contractors, and field
locations for updating EELs; and a lack of automation
in the EEL process. Problems with EELs are a top-5
•
30
JSF
 FOU, Alm.del - 2018-19 (1. samling) - Bilag 57: Henvendelse af 9/2-19 fra Heine Strømdahl, Gentofte, vedrørende F-35 problemer og den seneste rapport fra DOT og E
2014551_0009.png
FY18 DOD PROGRAMS
Not Mission Capable (NMC) maintenance driver and a
top-10 propulsion degrader for the U.S. Air Force.
▪ Users lack confidence in some ALIS functionality.
For example, the problems noted above have resulted
in users maintaining separate databases to track life
usage in case PAIRs erroneously generates incorrect
data. Users reference the external database created to
determine the correct values.
The timeline for correcting ALIS deficiencies is typically
excessive, causing workarounds to remain in place
for extended periods. For example, ALIS incorrectly
reports the status of aircraft as NMC in the Squadron
Health Management application based on HRCs
(faults). Meanwhile, a separate application – Customer
Maintenance Management System, which relies on
the Mission Essential Function List (MEFL) – reports
the same aircraft as mission capable. A logistics test
and evaluation report for ALIS version 1.0.3A3 in
December 2012 first noted this problem, yet it remains
today in ALIS 3.0.1.2.
▪ Many open deficiencies were not resolved during SDD
and will continue to negatively affect aircraft availability
and SGR.
▪ During SDD, the program repeatedly demonstrated
that attempting major software releases with large
increments of ALIS capability resulted in delays and
deferring capability. The program also did not allocate
sufficient resources to simultaneously develop new
required capabilities and reduce technical debt. Smaller,
more frequent releases would allow the program to field
new capabilities and fixes and receive frequent user
feedback to plan for future improvements, which the
program plans to do in C2D2.
The program has completed several deployments to
established bases and to austere locations and ships. In
each location, the complexities of ALIS have caused a
variety of information technology problems that delay
the unit’s ability to start generating sorties. Often, the
timeframe to start flight operation is longer than that with
legacy aircraft.
The program plans to release an updated version of
ALIS software (ALIS 3.1) to the international partners
and foreign military sales customers that includes
country-unique data (a.k.a. sovereign data) management
within ALIS beginning in January 2019 .
The program plans an additional major release of ALIS
software, version 3.5, scheduled for fielding in mid-2019,
during IOT&E. ALIS 3.5 will be a stabilization release,
since it is intended to address a large amount of technical
debt, meet cybersecurity threshold requirements –
including the use of internet protocols, improving LOHAS,
and providing an initial centralized capability for ALIS
administration. The program plans to complete ALIS 3.5
with SDD funds.
The program currently plans two additional releases,
ALIS 3.6 and 3.7, to provide additional stabilization and
improved sortie generation capabilities.
▪ ALIS 3.6, scheduled for release in mid-2020, is planned
to include Windows 10, additional cybersecurity
enhancements, improved air vehicle data transfers
between SOUs, and a decentralized maintenance
capability, which would allow deployments without a
full suite of ALIS hardware. The program also plans
to replace obsolescent hardware with the rollout and
fielding of the ALIS 3.6 software.
▪ The goal of ALIS 3.7, planned for release in mid-2021,
is improved mission support by adding capability to the
Training Management System, improved spare parts
support for deployments, support for partial squadron
deployments, corrosion management, and ALIS support
for helmets and other pilot flight equipment.
▪ Because EELs is a top degrader, the program is working
on high-priority corrective actions. However, per the
JPO, the software capabilities planned for ALIS 3.7 will
not address the root causes of the enterprise issues. This
is an excessive delay for needed fixes.
- The release plan for ALIS 3.5 through 3.7 shows the
program is moving toward a pace of one major software
release per year with fielding of service packs between
major releases. The program has demonstrated that it has
difficulty fielding large increments in ALIS capability.
While this movement toward more agile software
development is positive, the JPO will need to provide
sufficient resources for this effort.
- The use of ALIS across the F-35 enterprise would improve
data integrity as contractors and vendors would be required
to adhere to EELs requirements earlier in production and
sustainment.
▪ Lockheed Martin did not use ALIS in its production
facilities until recently, adding an SOU to the factory
floor in March 2018, shortly before propulsion system
installation, to improve data quality.
▪ Because data problems are frequently found when new
aircraft arrive at operational locations, Lockheed Martin
plans to begin using an SOU on the Fort Worth flight
line in early 2019 to support aircraft before delivery.
▪ While the addition of SOUs to the production line is a
positive step in addressing data problems, the program
will not extract maximum benefit from this effort unless
ALIS is fully integrated into production facilities.
▪ Vetting the data accompanying spare parts provided by
suppliers in an SOU before allowing delivery to field
units will reduce EELs deficiencies.
- Assessment of the testing regimen for ALIS.
▪ The program still relies heavily on the results of
laboratory testing of ALIS software, which does not
resemble operational conditions in several ways,
including the limited amount of data processed and
external connections.
▪ After the problems found during ALIS 2.0.2.4 testing
and fielding, the program moved toward heavier use
of ALIS testing facilities at Edwards AFB. However,
these test venues do not permit testing of the full range
-
-
-
-
-
JSF
31
 FOU, Alm.del - 2018-19 (1. samling) - Bilag 57: Henvendelse af 9/2-19 fra Heine Strømdahl, Gentofte, vedrørende F-35 problemer og den seneste rapport fra DOT og E
2014551_0010.png
FY18 DOD PROGRAMS
of ALIS capabilities. A single ALIS test venue would
increase test efficiency and support more timely fielding
of ALIS software to operational units. In the meantime,
the program uses an operational assessment process at
Nellis AFB to evaluate ALIS software releases before
deployment to the rest of the fleet.
▪ The current, non-operationally representative method
of testing ALIS releases leads to delays in finding and
fixing deficiencies, often after the new software is
fielded.
▪ Differences in laboratory testing and fleet personnel
procedures show that fleet personnel use ALIS
differently than the laboratory testers. Developmental
testing, particularly laboratory-based testing, should
include a variety of personnel from different Services
and experience levels to increase the chances of finding
problems early.
▪ ALIS testing, architecture, operations, and fielding each
absorb a disproportionate amount of time, manpower,
and funding. The program is developing automated
testing capabilities that are being accelerated in an
attempt to improve lab testing speed and quality.
Cybersecurity Operational Testing
• Activity
- The JOTT continued to accomplish testing based on
the cybersecurity strategy approved by DOT&E in
February 2015. The JOTT assessed F-35 training systems,
the ALIS-to-shipboard network interface onboard a
nuclear-powered aircraft carrier (CVN) with ALIS 2.0.2,
and ALIS version 3.0.
- The JOTT tested ALIS 3.0 at all three levels of operation:
▪ Autonomic Logistics Operating Unit (ALOU)
▪ Central Point of Entry (CPE)
▪ Squadron Kit (SQK), composed of the SOU, the
Mission Planning and Support Boundary, and the Low
Observable Maintenance Boundary
- In September 2018, the JOTT conducted Cooperative
Vulnerability and Penetration Assessments (CVPAs) of
ALIS 3.0.1.1 using National Security Agency-certified
cybersecurity test organizations and personnel:
▪ The Air Force’s 346 Test Squadron assessed the sole
ALOU at Lockheed Martin, Fort Worth, Texas.
▪ The Air Force’s 47 Cyber Test Squadron (CTS) assessed
the sole U.S. CPE at Eglin AFB, Florida, and the SQK at
Edwards AFB, California.
- In October 2018, the JOTT conducted Adversarial
Assessments (AAs) of the next iteration of ALIS 3.0
software – version 3.0.1.2 – with the assistance of National
Security Agency-certified Red Teams.
▪ The Marine Corps Red Team (MCRT) assessed the
ALOU.
▪ The Air Force’s 57 Information Assurance Squadron
(IAS) assessed the CPE.
▪ The Air Force’s 177 IAS assessed the SQK at Edwards
AFB, California.
- The ALIS 3.0 AA also included a limited Enterprise
Assessment of the boundaries and interfaces between
the ALOU, CPE, and SOU; Lockheed Martin Red Team
testing of the Lockheed Martin Internal network, with
observation by U.S. Government cyber test personnel; and
a preliminary investigation into the cybersecurity posture
of the supply chain for components of the SQK.
- The JOTT tested the three different network environments
present at the Academic Training Center at Eglin AFB,
Florida:
▪ The Unclassified Operating Environment (UOE),
consisting of unclassified classroom and training
resources.
▪ The Classified Operating Environment (COE),
consisting of classified classroom and training resources.
▪ The Full Mission Simulator (FMS), consisting of
pilot training stations for rehearsing mission tasks in a
simulated cockpit.
- In February through April 2018, the JOTT conducted
CVPAs of the UOE, COE, and FMS respectively in
partnership with the 47 CTS.
- In April 2018, the JOTT conducted AAs of the UOE and
COE utilizing the 57 IAS.
- In July 2018, the JOTT conducted an AA of the FMS with
the assistance of the 177 IAS.
- In August 2018, the JOTT conducted an AA onboard the
USS
Abraham Lincoln
of the network interface between
a deployed SQK in the ALIS 2.0.2 configuration and
the ship’s Consolidated Afloat Networks and Enterprise
Services internal network. The MCRT also facilitated the
test.
- All JSF cyber tests in 2018 were completed in accordance
with their individual, DOT&E-approved test plans.
- Throughout 2018, the JOTT continued to work with
stakeholders across the DOD to identify relevant scenarios,
qualified test personnel, and adequate resources for
conducting cyber testing on air vehicle components and
systems.
- The JOTT expects to conduct a CVPA and AA of
the USRL in early 2019, as well as several cyber
demonstrations involving air vehicle components and
sub-systems.
•
Assessment
- Cybersecurity testing in 2018 showed that some of the
vulnerabilities identified during earlier testing periods still
had not been remedied.
- More testing is needed to assess the cybersecurity of the
air vehicle. Actual on-aircraft or appropriate hardware-
and software-in-the-loop facilities are necessary to enable
operationally representative air vehicle cyber testing.
- Testing of the JSF supply chain to date has not been
adequate. Additional testing is needed to ensure the
32
JSF
 FOU, Alm.del - 2018-19 (1. samling) - Bilag 57: Henvendelse af 9/2-19 fra Heine Strømdahl, Gentofte, vedrørende F-35 problemer og den seneste rapport fra DOT og E
2014551_0011.png
FY18 DOD PROGRAMS
integrity of hardware components for initial production
of air vehicles and ALIS components, plus resupply of
replacement parts.
- Testing to date has identified vulnerabilities that must be
addressed to ensure secure ALIS operations.
- According to the JPO, the air vehicle is capable of
operating for up to 30 days without connectivity to ALIS.
In light of current cybersecurity threats and vulnerabilities,
along with peer and near-peer threats to bases and
communications, the F-35 program and Services should
conduct testing of aircraft operations without access to
ALIS for extended periods of time.
Availability, Reliability, and Maintainability
•
Activity
- The program continued to deliver aircraft to the U.S.
Services, international partners, and foreign military sales
throughout CY18 in production Lot 10. As of the end of
September, 323 operational aircraft had been produced
for the U.S. Services, international partners, and foreign
military sales. These aircraft are in addition to the 13
aircraft dedicated to developmental testing.
- As of the end of June, the U.S. fleet of F-35s had
accumulated 126,136 flight hours
- The following assessment of fleet availability, reliability,
and maintainability is based on sets of data collected from
the operational and test units and provided by the JPO.
The assessment of aircraft availability is based on data
provided through the end of August 2018. Reliability and
maintainability assessments in this report are based on
data covering the 12-month period ending June 30, 2018.
Data for reliability and maintainability include the records
of all maintenance activity and undergo an adjudication
process by the government and contractor teams, a process
which creates a lag in publishing those data. The variety
of data sources and processes are the reasons the data have
different dates and appear to be delayed.
•
Assessment
- The operational suitability of the F-35 fleet remains at
a level below Service expectations. Similar to the 2017
DOT&E report, most suitability metrics remained nearly
the same throughout 2018 or moved only within narrow
bands.
- Aircraft availability is determined by measuring the
percentage of time individual aircraft are in an “available”
status, aggregated monthly over a reporting period.
▪ The program-set availability goal is modest at
60 percent, and the fleet-wide availability discussion
uses data from the 12-month period ending August 2018.
▪ For this report, DOT&E is reporting availability rates
only for the U.S. fleet, vice including international
partner and foreign military sales aircraft, as was done in
previous reports.
- The fleet-wide monthly availability rate for only the U.S.
aircraft, for the 12 months ending August 2018, is below
the target value of 60 percent. The DOT&E assessment of
the trend shows no evidence of improvement in U.S. fleet
wide availability during 2018 .
- Aircraft that are not available are designated in one
of three status categories: Not Mission Capable for
Maintenance (NMC-M), Depot (in the depot for
modifications or repairs beyond the capability of unit-level
squadrons), and Not Mission Capable for Supply
(NMC-S).
▪ The average monthly NMC-M and Depot rates were
relatively stable, with little variability, and near program
targets.
▪ The average monthly NMC-S rate was more variable,
and was higher (i.e., worse) than program targets .
▪ The average monthly utilization rate measures flight
hours per aircraft per month. The average utilization
rate of flight hours per tail per month increased slightly
over previous years, but remains below original Service
bed down plans.
▪ The low utilization rates continue to prevent the Services
from achieving their programmed fly rates, which are
the basis of flying hour projections and sustainment
cost models. As of June 30, 2018, the fleet had flown
126,136 hours. This amounted to 83 percent of an
early 2017 “modeled achievable” projection of 152,445
flight hours by the end of June, 2018. Similarly, for the
12 months ending April 2018, the U.S. Services had
contracted for 42,836 flight hours, but the U.S. F-35
fleet logged only 33,365 hours, or 78 percent of the
contracted amount over this period.
- A separate analysis of availability of the OT-instrumented
fleet, using data from the 12-month period ending
August 2018, is important to consider now that formal
IOT&E is underway. The numbers below account for
the full complement of 23 U.S. and international partner
aircraft assigned to the OT fleet at the end of August 2018
(8 F-35A, 9 F-35B, and 6 F-35C).
▪ The average monthly availability rate for F-35 OT
aircraft was below the planned 80 percent needed for
efficient conduct of IOT&E. The low availability
during this period is partly explained by the fact that
the aircraft of the OT fleet spent over a quarter of the
time in depot modifications to bring them up to the
Lot 9 production-representative standard configuration,
as required prior to the start of IOT&E, with some
DOT&E-approved modification deferrals.
▪ Availability of the OT fleet will remain a challenge for
the efficient conduct and timely completion of IOT&E.
Although the necessary modifications have been
completed on the OT aircraft and formal testing has
started, mission capable aircraft will need to be available
at a high rate to complete the open-air test trials as
scheduled.
JSF
33
 FOU, Alm.del - 2018-19 (1. samling) - Bilag 57: Henvendelse af 9/2-19 fra Heine Strømdahl, Gentofte, vedrørende F-35 problemer og den seneste rapport fra DOT og E
2014551_0012.png
FY18 DOD PROGRAMS
F-35 Fleet Reliability
- Aircraft reliability assessments include a variety of
metrics, each characterizing a unique aspect of overall
weapon system reliability.
▪ Mean Flight Hours Between Critical Failure (MFHBCF)
includes all failures that render the aircraft unsafe to fly,
along with any equipment failures that would prevent
the completion of a defined F-35 mission. It includes
failures discovered in the air and on the ground.
▪ Mean Flight Hours Between Removal (MFHBR)
indicates the degree of necessary logistical support
and is frequently used in determining associated costs.
It includes any removal of an item from the aircraft
for replacement. Not all removals are failures; some
removed items are later determined to have not failed
when tested at the repair site, and other components
can be removed due to excessive signs of wear before a
failure, such as worn tires.
▪ Mean Flight Hours Between Maintenance Event
Unscheduled (MFHBME_Unsch) is a reliability metric
for evaluating maintenance workload due to unplanned
maintenance. Maintenance events are either scheduled
(e.g., inspections or planned part replacements) or
unscheduled (e.g., failure remedies, troubleshooting,
replacing worn parts such as tires). MFHBME_Unsch
is an indicator of aircraft reliability and must meet
the Operational Requirements Document (ORD)
requirement.
▪ Mean Flight Hours Between Failure, Design
Controllable (MFHBF_DC) includes failures of
components due to design flaws under the purview of
the contractor, such as the inability to withstand loads
encountered in normal operation.
- The F-35 program developed reliability growth projection
curves for each variant throughout the development
period as a function of accumulated flight hours. These
projections compare observed reliability with target
numbers to meet the threshold requirement at maturity
(200,000 total F-35 fleet flight hours, made up of 75,000
flight hours each for the F-35A and F-35B, and 50,000
flight hours for the F-35C). As of June 30, 2018, the date
of the most recent set of reliability data available, the
fleet and each variant accumulated the following flight
hours, with the percentage of the associated hour count at
maturity indicated as well:
▪ The complete F-35 fleet accumulated 126,136 flight
hours, or 61 percent of its maturity value.
▪ The F-35A accumulated 74,758 hours, or over
99 percent of its maturity value.
▪ The F-35B accumulated 35,076 hours, or 47 percent of
its maturity value.
▪ The F-35C accumulated 16,302 hours, or 33 percent of
its maturity value.
- The program reports reliability and maintainability metrics
for the 3 most recent months of data. This rolling 3-month
window dampens month-to-month variability while
providing a short enough period to distinguish current
trends.
- Table 1 shows the trend in each reliability metric by
comparing values from May 2017 to those of June 2018
and whether the current value is on track to meet the
requirement at maturity.
TABLE 1. F-35 RELIABILITY METRICS (UP ARROW REPRESENTS IMPROVING TREND)
Assessment as of June 30, 2018
Flight
Hours
for ORD
for JCS
Threshold
MRHBCF (Hours)
Cumulative
Flight
Hours
Change:
May
2017
to June
2018
Meeting
Interim
Goal
for ORD
Threshold
MFHBR (Hours)
Change:
May
2017
to June
2018
Meeting
Interim
Goal
for ORD
Threshold
MFHBME (Hours)
Change:
May 2017
to June
2018
Meeting
Interim
Goal
for ORD
Threshold
MFHBF_DC (Hours)
JCS
Require-
ment
Change:
May 2017
to June
2018
Meeting
Interim
Goal
for ORD
Threshold
Variant
ORD
Threshold
ORD
Threshold
ORD
Threshold
F-35A
75,000
74,758
20
↑
No
6.5
↑
No
2.0
No
Change
↑
↑
No
6.0
↑
Yes
F-35B
F-35C
75,000
50,000
35,076
16,302
12
14
↑
↓
No
No
6.0
6.0
↑
↓
No
No
1.5
1.5
No
No
4.0
4.0
↑
↑
Yes
Yes
- Between May 2017 and June 2018, six of the nine ORD
metrics increased in value, often marginally, two decreased
marginally, and one remained the same. Consistent with
previous reports, the three JSF Contract Specification
(JCS) metrics continued to show the strongest growth
and, in all cases, were above their specifications for the
3 months ending June 2018. This strong MFHBF_DC
growth has still not translated into equally strong growth
for the ORD reliability metrics, all of which fall short of
their interim goals.
- More in-depth reliability growth analyses conducted
by DOT&E show that the ORD reliability metrics are
growing, albeit slowly, especially for F-35B and F-35C
MFHBCF. Also, for the majority of the metrics, reliability
grew markedly more slowly after the release of the
Block 2B flight envelope than before. Based on these
34
JSF
 FOU, Alm.del - 2018-19 (1. samling) - Bilag 57: Henvendelse af 9/2-19 fra Heine Strømdahl, Gentofte, vedrørende F-35 problemer og den seneste rapport fra DOT og E
2014551_0013.png
FY18 DOD PROGRAMS
analyses, none of the ORD metrics are predicted to meet
their requirements by their individual variant maturity
milestones.
- In addition to reporting the MFHBCF values above,
the JPO adopted a second, alternative approach for
reporting MFHBCF in 2017 that only counts critical
failures that take 8 hours or more to remedy. This
approach presumably supports modeling of SGR, a Key
Performance Parameter in the ORD.
▪ DOT&E continues to disagree with this approach
because failures that take less than 8 hours to remedy
will likely still affect SGR, especially during a combat
sortie surge. Also, it is not consistent with the widely
accepted definition of the MFHBCF measure.
Maintainability
- The amount of time needed to repair aircraft and return
them to flying status has changed little over the past year,
and remains higher than the requirement for the system
at maturity. The program assesses this time with several
measures, including Mean Corrective Maintenance Time
for Critical Failures (MCMTCF) and Mean Time To
Repair (MTTR) for all unscheduled maintenance. Both
measures include “active touch” labor time and cure times
for coatings, sealants, paints, etc., but do not include
logistics delay times, such as how long it takes to receive
shipment of a replacement part.
▪ MCMTCF measures active maintenance time to correct
only the subset of failures that prevent the F-35 from
being able to perform a specific mission. It indicates the
average time for maintainers to return an aircraft from
NMC to MC status.
▪ MTTR measures the average active maintenance time
for all unscheduled maintenance actions. It is a general
indicator of the ease and timeliness of repair.
- The program reports maintainability metrics for the 3 most
recent months of data. Table 2 shows the nominal change
in each maintainability metric by comparing values from
May 2017 to those of June 2018, and whether the current
value is on track to meet the requirement at maturity.
▪ All mean repair times are longer, some up to more
than twice as long, as their ORD threshold values for
maturity, reflecting a heavy maintenance burden on
fielded units.
- The JPO, after analyzing MTTR projections to maturity,
acknowledged that the program would not meet the
MTTR requirements defined in the ORD. The JPO is
seeking relief from the original MTTR requirements and
has proposed new values of 5.0 hours for both the F-35A
and F-35C, and 6.4 hours for the F-35B. This will affect
the ability to meet the ORD requirement for SGR, a Key
Performance Parameter.
TABLE 2. F-35 MAINTAINABILITY METRICS (DOWN ARROW REPRESENTS IMPROVING TREND)
Assessment as of June 30, 2018
MCMTCF (Hours)
Variant
Flight Hours for ORD
Threshold
Cumulative Flight
Hours
Change: May
2017 to June
2018
Meeting Interim
Goal for ORD
Threshold
MTTR (Hours)
Change: May
2017 to June
2018
Meeting
Interim Goal for
ORD Threshold
ORD Threshold
ORD Threshold
F-35A
75,000
74,758
4.0
↓
No
2.5
↓
No
F-35B
F-35C
75,000
50,000
35,076
16,302
4.5
4.0
↑
↓
No
No
3.0
2.5
↑
↓
No
No
Live Fire Test and Evaluation
F-35 Vulnerability to Kinetic Threats
•
Activity
- In April 2018, Lockheed Martin delivered the F-35
Vulnerability Assessment Report summarizing the force
protection and vulnerabilities of all three F-35 variants,
and the F-35 Consolidated LFT&E Report, which
summarizes the live fire test and analysis efforts supporting
the vulnerability assessments.
•
Assessment
- The assessments conclude the following:
▪ For three of the four specification threats, the F-35
variants meet JSF contract specification requirements
to enable safe ejection of the pilot in the event of an
engagement.
▪ For two of the four specification threats, the F-35A
and F-35C variants meet JSF contract specification
requirements to return safely to the Forward Line of
Troops (FLOT) following an engagement. The F-35B
met the requirements for only one of the four threats.
▪ All three F-35 variants are less vulnerable to three of the
four specification threats than the legacy F-16C aircraft,
both for safe ejection and for return to FLOT.
- DOT&E will publish an independent evaluation of the
vulnerabilities of the F-35 aircraft variants to expected
JSF
35
 FOU, Alm.del - 2018-19 (1. samling) - Bilag 57: Henvendelse af 9/2-19 fra Heine Strømdahl, Gentofte, vedrørende F-35 problemer og den seneste rapport fra DOT og E
2014551_0014.png
FY18 DOD PROGRAMS
and emerging threats in the report to support the Full-Rate
Production decision scheduled for FY20.
F-35 Vulnerability to Unconventional Threats
•
Activity
- As of FY17, the Naval Air Warfare Center Aircraft
Division at NAS Patuxent River, Maryland, completed
full-up system-level testing of F-35A and C variants,
and limited testing of the F-35B, to evaluate tolerance to
electromagnetic pulse threats.
- The program completed full-up, system-level,
chemical-biological decontamination testing on
BF-40 (a low-rate initial production F-35B aircraft) in
February 2017.
•
Assessment
- Testing was done to the threat level defined in Military
Standard 2169B. Follow-on, full-up, system-level tests
of the F-35B, including a test series to evaluate Block 3F
hardware and software changes, are ongoing.
- In the event of a chemical or biological attack, the
equipment is capable of decontaminating the F-35.
Additional work would be needed to develop an
operational decontamination capability.
▪ To assess the protection capability of the Generation
(Gen) II Helmet-Mounted Display System (HMDS)
against chemical-biological agents, the JPO completed
a comparison analysis of HMDS materials with those
in an extensive DOD aerospace materials database.
Compatibility testing of legacy protective ensembles and
masks showed that the materials used in the protective
equipment can survive exposure to chemical agents and
decontamination materials and processes. The program
plans similar analyses for the Gen III and Gen III Lite
HMDS designs. While this assessment of material
compatibilities provides some understanding of the force
protection capability against chemical and biological
agents, it does not demonstrate the process required to
decontaminate either HMDS.
F-35 Gun Lethality
•
Activity
- From August through December 2017, during DT Weapons
Delivery Accuracy testing, the Naval Air Warfare Center
Weapons Division at Naval Air Weapons Station China
Lake completed air-to-ground flight lethality tests of three
different 25 mm ammunitions including the PGU-32/U
Semi-Armor-Piercing High-Explosive Incendiary
round, PGU-47/U Armor-Piercing High-Explosive
Incendiary with Tracer round, and PGU-48/B Frangible
Armor-Piercing round. Flight lethality tests included gun
firings from all three F-35 variants against armored and
technical vehicles, small boats, and plywood manikins.
Tests revealed deficiencies with the Armor-Piercing
High-Explosive round’s fuze reliability for impacts into
the ground. Nammo, the Norwegian manufacturer, is
conducting testing to further modify the fuze design and
increase reliability.
•
Assessment
- The weapon-target-pairing lethal effects are currently
being analyzed by DOT&E.
Recommendations:
• The program should:
1. Continue to work with the Services to prioritize and correct
the remaining Category 1 and 2 deficiencies discovered
during SDD.
2. Apply lessons learned from SDD and other programs for
scoping the amount of C2D2 testing that can be done in
laboratories and simulations, compared with the need for
flight testing.
3. Reassess the C2D2 plan to ensure adequate test
infrastructure (labs, aircraft, and time) is provided and
modifications are aligned with other fielding requirements.
4. Assess the annual cost of software sustainment.
5. Determine the cause of the accuracy problems with the
F-35A gun firing and implement a solution for increasing
gun accuracy for the fielded aircraft.
6. Develop a consolidated and adequate ALIS test venue to
ensure ALIS capabilities are fully tested prior to fielding to
operational units
7. Conduct a study to determine the optimum balance of
additional spare parts procurement versus adding depot
capacity to repair spare parts, in order to decrease the
percentage of NMC aircraft waiting for spare parts.
8. Continue implementing measures to improve fleet
availability.
9. Make actual aircraft or appropriate hardware- and
software-in-the-loop facilities available to enable
operationally representative air vehicle cyber testing.
10.Continue conducting periodic rounds of cybersecurity
testing and correcting open cyber deficiencies.
11. Continue testing the integrity and security of the JSF supply
chain, expanding on initial testing conducted in 2018.
• The JPO should:
1. Complete contracting actions to procure a second F-35B
ground test article in order to complete at least two lifetimes
of structural durability testing to validate the wing-carry-
through structure.
2. Fund and contract for the 16-20 recommended signal
generators called for in the JPO’s own 2014 gap analysis
study.
3. Fund and contract for the necessary hardware upgrades to
the USRL to support Block 4 development and testing.
36
JSF