Authors: David Hitt,Heather R. Smith
We ended up realizing that we had built an electric airplane that had essentially only one operating flight control system. So we said, “Well, what if we’re wrong? No one has ever flown a Mach 20 airplane. This whole flight envelope is something that nobody’s ever had the opportunity to experience. So what do you suppose our tolerance is to this?” Because wind tunnel models for the ascent vehicles, they fit in your hand, because the tunnels that were able to handle these things were small. The wind tunnel models for the orbiter were larger, but they’re still not all that big, and going through this tremendously wide flight regime where the air density is going from nothing to everything, and it’s just
high speeds to low speeds, I said, “What’s the chance of getting all that right?” And yet as we played in these simulators, . . . we proved to ourselves that, boy, if you’re off on that estimate of the aerodynamics, you can often play with the software to make it right, but if the real aerodynamics and the software you have don’t match, it’s a real mess. I know I worried a lot about that.
So we came up with a concept that we would have some tolerances on the aerodynamics, and we would try to make sure that the flight control system could handle these kind of uncertainties in aerodynamics. We did something which is not typically done—we decided to optimize the flight control performance to be tolerant on uncertainties rather than the best flight control system they could build. The whole idea was, after we’ve flown and we have some experience and we know what the real world is, now we can come back and make it better, but the first job is to make ours as tolerant as possible to the things we don’t know.
While Mattingly was working with the computer models of the flight dynamics of the shuttle, astronaut Hank Hartsfield was on the other side of that research, working with the wind tunnel models and encountering the same concerns about the scalability of the data coming out of those tests.
As I recall, the shuttle program had over twenty-two thousand hours of wind tunnel time to try to figure out what it flies like. Because the decision had been made, there are no test flights. We were going to fly it manned the first flight, and an orbital flight at that, which demanded that, the best you can, [we] understand this. Well, hypersonic aerodynamics is difficult to understand, the uncertainty on the aerodynamic parameters that you get out of the tunnel are big. The things that we were looking at in the simulations were if these uncertainties in the different aerodynamic parameters stack in a certain way, the vehicle could be unstable.
What we were looking for, for those combinations, statistically were possible, but hopefully not very probable they’d happen, but if they did, that was the kind of things we had to plan for. It’s just an uncertain world. You can’t predict, because in the wind tunnel, you have to put in scaling factors. If you’re doing wind tunnel things off a small model, it doesn’t really scale to the big model perfectly, and you have to make assumptions when you do that. The scaling ratios have a big factor, a big effect on what the real numbers are. So if you could fly a full-scale orbiter in the wind tunnel and it would go Mach 15 or something, it would be great, but you can’t do that. You have a little-bitty model, and it’s a shock tunnel or something. You’d get a few seconds of runtime at the right Mach numbers and then try to capture the data off of that.
Space Shuttle vehicle testing in the fourteen-foot Transonic Wind Tunnel at
’s Ames Research Center. Courtesy
Astronaut Don Peterson was involved in studying the redundancy of systems on the orbiter, and particularly the flight control computers. In the report he pointed out that failure rates on some of the avionics could be high. On Apollo and earlier vehicles,
built “ultra-reliability components,” components that were overdesigned and tested to make failures less likely.
Failures on Apollo, for that reason, were pretty rare. But that’s very expensive. That’s a very difficult thing to do. I was told that after the lunar program ended,
had two of the lunar module computers left over, spares. So they just turned them on and programmed them to run cyclically through all the programs. I think they ran one of those computers for, like, fifteen years, and it never failed. It just kept running, and finally they turned it off. They just said, “It’s not ever going to fail.” That’s the way that equipment was built. But that makes it very
expensive. So when they built the Shuttle, they said, “We can’t do that. So what we’re going to do is, instead of ultrareliability components, we’re going to rely on something called redundancy.” They were going to have four computers, and they were going to have three
s, and they were going to have four of this and two of that and so on. That way, you could tolerate failures. But as a result of that, the failure rate on some of that equipment was fairly high, compared to Apollo.
They also made the multiple units interdependent. “On a typical automobile you have five tires, but that’s not five levels of redundancy because you need four of them,” Peterson explained.
So you can really only tolerate one failure. You can have one tire go bad and you can take care of that. But we got into that same situation on the shuttle because of the way they did the software. The shuttle, when it’s flying, the computers all compare answers with one another, and then they vote among themselves to see if anybody’s gone nuts. If a computer has gone bad, the other computers can override its output so that it isn’t commanding anything. But to make that scheme work, you have to have at least three computers working. Otherwise, you can’t vote. You could have [two systems voting], but if they vote against each other, you don’t know which one’s the bad one.
The decision was made to put five of the computers on the orbiter, with four of them active in the primary system, with the idea that this would create a system that could tolerate three failures. However, Peterson said, this produced much higher failure rates than expected. While the system provided a high amount of redundancy in theory, the reality was that because of the way it was designed, the system actually could tolerate only one failure safely. The four primary computers were not truly redundant for each other; only the spare provided redundancy. If one computer failed, the spare would take its place. After that, however, further failures would endanger the cooperative “voting logic” between the computers that verified the accuracy of their results.
But the complexity of the way the thing was put together kind of defeated the simplistic redundancy scheme that they had. It’d be like driving a car that had two engines or three engines, and any one of them would work. Well, that way you could fail two engines and you’d still drive right along. But if it takes two engines to power the vehicle, then you don’t have that, and if it takes three en
gines to power the vehicle, you don’t have any redundancy at all. It gets to be a game then as to how you trade all this off. When I looked at all that and we put the study together, we said, “You know, you’re going to have some failures that are going to really bother you because you’re going to lose components.” For example, you’re on orbit and you’ve got four computers and one of them fails. Well, now you’ve got three computers left in the primary set. But do you stay on orbit? Because if you suffer one more failure, your voting algorithm no longer works. Now you’re down then into coming home on a single computer and trusting it. And nobody wanted to do that.
So they said, “Gee, I’ve got four computers. I can only tolerate one failure, and then I’ve got to come home.” We had four of some of the other components, and it was kind of the same sort of thing. If one of them fails, we are no longer failure tolerant. We’ve lost the capability to compare results and vote, and so we don’t want to stay on orbit that way. So now, all of a sudden, the fact that you’ve got four of them causes more aborts because the more things you have, the more likely you are to have one fail. You’d get more failures and more aborts with four computers than if you’d gone with some other plan. That was pretty controversial for a while. We predicted—and there were some people that were really upset about that—we predicted a couple of ground aborts due to computer failures. Essentially we’d get chewed out for saying that, but in the first thirteen flights, we hit it right on the money. We had two ground aborts in thirteen flights.
When the shuttle was built, the air force was also using redundancy systems, Peterson recalled. Then the air force built what it called confederated systems, in which each component was independent. “They cooperated with each other, but they shipped data to each other, but they weren’t really closely tied together,” Peterson explained.
The shuttle was tightly integrated. It runs on a very rigid timing scheme. The computers on the shuttle actually compare results about a little more than three hundred times a second. So it’s all tightly tied together. Well, when they decided to build the [International] Space Station,
said, “We’re not doing this integrated stuff anymore. Boy, that was a real pain. We’re going to use a confederated system.” The air force, on their latest fighter, said, “This confederated stuff doesn’t work worth a damn. We’re going to build a tightly integrated [system].” So they both went along for ten years or twelve years, and then they flip-flopped. The military’s going the way
originally went, and
’s now going the way
the military went originally. I think the answer is, there is no magic answer to all that. Probably one concept is maybe not that much better than the other. It’s how you implement it and how much money you spend and how much to test. What do they say? The devil’s in the details. I think that’s right with all this stuff.
Mattingly recalled excellent cooperation between the engineering staff working on the shuttle and the Astronaut Office. “I seldom have seen that integration of the people that were going to fly it with the designers and people who were doing the theoretical work and the operators from the ground,” Mattingly said.
All of that stuff was converged in parallel, and I think that’s one of the reasons that the shuttle is such a magnificent flying machine. It does all the magic that we set out to do. I’m ignoring the cost because the shuttle, in my recollection, by the time it was sold to Congress, it was probably different than what the people in the trenches remember, but we had to do all these technical things, and it was a matter of faith that if you build it, it will be cheap. I mean, it was just simple. If you could reuse it, it saves money, and so you’ve got to make it reusable. If you fly a lot, that will be good, and we’re going to fly this thing for $5.95, and we’re going to fly it once a week and that’s how we’re going to do this. And none of us were ever told to go build a vehicle that we could afford to own. And had we been told that, I doubt if we would have been able to do it. I think the job was so complex, you had to build one that flies in order to learn the lessons that say, “Now I know what’s important and what isn’t.” I just think it would have been asking too much, but that’s just personal opinion, but it’s from having struggled through ten years of this development program. It was an extraordinary experience to do that.
The role of the Astronaut Office during the development of the Space Shuttle was quite different from what Mattingly experienced during the Apollo program. “Our involvement was far more extensive and pervasive, and a heck of a lot more fun,” he said.
I mean, this was really cool stuff. There was a problem every day, and you got to learn about all of these little things that were interesting. I spent a lot of time trying to understand the stress loads and the thermal characteristics on the
[thermal protection system], and how do you get it to stay on, and all of those things were things that came through the office as experiences that really were
just extraordinary opportunities to go see that. As we moved down the stream and we got into some of these development programs and started turning out hardware, we started splitting people up to go follow different components of hardware, whether it be the engines or the
s or the orbiter.
The decision to have the orbiter be an unpowered glider rather than a jet during its return to Earth and the various ramifications of that decision were also among the things that had to be considered during development. “Somewhere earlier in this development stage, we went through a series of activities where the first orbiter was going to have air-breathing engines, and it had some solid rockets that were on the back that were for aborts,” Mattingly said.
Right off the pad you could fire these two big rockets, and they would take you off in a big loop so you could come back and land. We had these air-breathing engines that were going to—after you come down through the atmosphere, you open the door and these engines come out, and you light them and you come around and land. They had enough gas for one go-around. The other thing we had was the big solids were to have thrust terminations and ports that blew out at the front end so you could terminate thrust on them if you needed to in an emergency. Every one of those devices was something which had a higher probability of killing you by its presence than it would ever have in saving you. I’ll put that ejection seat in the same boat. Everybody was willing to get rid of the air-breathing engines. They were really, really not a very bright idea. And we got rid of the thrust termination and we got rid of the abort solid rockets. My guess is John Young was probably the most active stimulus in pushing those issues, and that was one of those cases where the flight crew perspective and the engineering perspectives converged. We all wanted to get rid of these things, and yet we retained the ejection seats for reasons which I will never understand. If anyone knew what the useful envelope of those ejection seats was and the price we paid to have them. . . . But it had become a cause: “You will protect these kids by giving them an ejection seat.” So we had one, not that anybody wanted to ever use it, but it was there.