Computer Graphics: Helping to Cope With Terrorism
By Ben Delaney © 2002

This article originally appeared in IEEE Computer Graphics and Applications January/February © 2002

It was impossible for me to maintain the journalistic detachment that is a mark of professional reporting as I write this article. For I was one of thousands of New Yorkers crowded on rooftops, staring with disbelief, afraid to keep watching, and unable to look away. I saw the second plane hit, I saw the towers collapse, I imagined what was happening to the tens of thousands of people in those buildings and the buildings surrounding the towers. I took pictures. I had a stiff drink at ten in the morning. I cried. And I, like millions around the world, turned to television, and the graphic images presented there, to try to understand what was happening. -- Ben Delaney

The fist to the gut impact that graphics and images can have was driven home to millions of people around the world on a sunny Tuesday morning last September. September 11, 2001 is indeed a date that few alive today will ever forget. As graphics professionals, we can take some pride in the use of computer graphics technology in many aspects of the ensuing response. In an emergency, one doesn’t have time to develop new technology. Finding new ways to use what you have, leveraging the technology to the max, is what happens. This article will describe how CG imaging technology helped us comprehend the incomprehensible and saved lives, will help to memorialize the victims, and will assist in the rebuilding that is already being planned.

The words on the radio that morning were at first more curious than alarming; "a plane has struck the World Trade Center". But the images instantly broadcast around the world belied the mildness of that statement with pictures of fiery death and destruction beyond comprehension. When the second plane hit, live on TV, the images created reactions of horror and anger, because at that point there was no longer a question of an accident. The images – enhanced, replayed, slowed down and annotated – showed beyond doubt that this was a deliberate attack.

That morning, television did what it does best – it provided instant accessibility to dramatic imagery of the incidents, the other attack, on the Pentagon, and to hastily-created computer graphics that showed us the structural details of the buildings, and the flight paths of the planes. But the TV provided something else. It gave respite from the reality, a mile and a half from my rooftop, of towers burning in a clear Autumn sky.

This tragedy has called on high-tech and computing technology in novel ways, to make novel contributions. In every aspect of this event, computer graphics played a key role. Near-real-time 3D animation helped inform the world of what had happened, what hadn’t happened, and how America was dealing with the crisis. Mapping, sensor fusion, 3D visualization, and real-time coordination of GIS (Geographic Information System) data from various sources facilitated the rescue operations. Coordination of cleanup operations was aided by graphic analysis and planning tools. Even as the rescue operations were starting, artists from around the world were offering their designs for memorials for the WTC. Long before the cleanup efforts were complete, architects, planners, transportation planners, and other interested citizens contributed to, seeing, and commenting on plans for the site of the Towers and other buildings. An urban simulator developed to help small-town planners is being interfaced to New York City’s 3D GIS database to help create a new plan for Lower Manhattan.

While many of the applications of graphics were, in themselves, fairly ordinary, what is extraordinary is the overall contribution that CG made at every point in dealing with the attacks on the United States. We were better able to understand what had happened and how it was being dealt with. We were given instant history and geography lessons that relied heavily on imagery to get their message across. And our hopes for recovery are being visualized with stunning clarity as artists, architects, and planners use CG to communicate their visions for the future.

Inform

For most of us, the attacks of September 11 and their aftermath were witnessed on TV. Live, unedited, gripping, and awful footage rolled on our screens for days, uninterrupted by commercial messages. Of all that coverage, some of the most informative and interesting were the graphics used to explain the attacks, the buildings’ construction, and the rescue efforts.

Fox News Channel and the CNN are leaders in international news coverage. I, as did millions of other people around the world, watched their coverage of the attacks with horror, dismay, and disbelief. As we expected, the TV news people, especially those in New York City, many of whom had lost friends and family, did amazing work, live and on camera for days at a time.

The graphics departments were among the dozens of behind the scenes people that support the on-air talent. I talked to Richard O’Brien, Vice President/Creative director at FOX News Channel, and to Robert Hunter, Senior Design director at CNN. Each of them was responsible for getting up-to-date graphics on air in minutes, and each found innovative ways to accelerate the process.

Richard O’Brien was still at home in Connecticut when he learned of the attacks. Unable to get into Manhattan – all entrances and exits were closed minutes after the planes struck – he used his cell phone to stay in touch until late in the day, when he was the only passenger on the first train heading into the city. His first thought, he told me, was "what will the viewers want to see?"

The first response from the graphics department was to add the now-ubiquitous crawler, a ticker-tape-like text display overlay scrolling across the bottom of the screen. The first custom graphic, the "America Under Attack" crisis logo, was completed in an hour and a half after the attacks. Like most of FOX’s on-air graphics, these were sketched in Illustrator on Macintosh computers, then imported into Maya for 3D embellishment and animation. Within four or five hours, O’Brien’s team of around 25 graphics designers, illustrators, and animators, had created an animated sequence illustrating the attacks. To do this, they used models of 757s that they already had in the library, and 3D drawings of the twin towers that were done freehand, in Maya. As he explained, the towers’ design was pretty simple, so they were able to fake them at first. As time passed, the FOX teams cleaned up their first graphics, and created new ones on a 24/7 schedule that lasted six days.

In only a few hours TV news departments had crated animations illustrating the flight paths of the hijacked airliners. These frames, from an animation by Fox News Channel, were developed with library models of the plane and models of the buildings that were drawn by the art department from photos. They show the second plane striking WTC #2, the South Tower. Images courtesy Fox News Channel.

CNN was on a similar schedule, with news breaking every minute in the first few days. Like FOX News, they worked entirely in house, using much the same software; Alias|Wavefront’s Maya, Adobe’s Illustrator and PhotoShop, and Discreet’s flame and fire. Both networks used a combination of machinery, too; Macintosh G4’s, Windows graphics workstations, and SGI workstations and servers. The output of the graphics departments are either recorded on tape, or, more commonly, filed digitally (as Inventor files or in other applications’ native formats) in servers that are accessed directly by the technical directors in the control booth.

At CNN, broadcasts on the early days were almost entirely live feeds. Of course, they developed a logo for the event coverage, using Chyron’s Liberty paint and animation software. Their first graphics were largely aerial photos provided by Space Imaging Company. (SIC) (For security reasons after the military response started, the US Government stopped the supply of Space Imaging’s photos of Afghanistan by inking an agreement with SIC to exclusively buy all of their output.) These photos were imported into flame and fire, where zooms and pans were created for on-air use.

After the first weeks, when war preparations were occupying the Pentagon, CNN was also findings new ways to work on a war-time basis. Miles O’Brien, one of their most experienced reporters, teamed up with the graphics team in the weather department to use the special capabilities of the weather graphics system. This system, the Weather Services International (WSI) Weather Producer system, runs on SGI hardware and uses Inventor files as a primary format. But unlike the SGI workstations in the graphics department, the weather department’s systems are intended for real-time, interactive use. You’ve probably noticed the clicker many TV weather reporters carry. It is used to advance the WSI system, like a slide projector. Like PowerPoint, the WSI system can include animations, and links to FingerWorks’ Telestrator, so the reporter can make live annotations on screen. O’Brien realized that the WSI system, while not capable of producing the same high quality as the graphics department’s systems, was able to produce acceptable graphics and animations very quickly.

Robert Hunter explained that once the graphics people realized what O’Brien and the weather people had accomplished, they saw a way to make the on-air product even better. They started creating graphics on the SGI and NT workstations using Maya, and exported them as Inventor files that the WSI system could read. The result was high-quality stills, models, and animation, running interactively in real-time as O’Brien reported the war news.

Hunter said the motto in the first few days after the attacks was "First thing: make air. Second thing, make it look good". He described the steps CNN used to get 3D animations of vehicles, such as the planes used in the attacks, on air – often in just a few hours.
Find models of the vehicles. Viewpoint’s models were frequently used. Export those simple, minimally shaded models in Inventor format.
Create additional models in Maya, and export them as Inventor files.
Import the models into flame or fire. In those programs, detailing was applied, and the models were composited with maps, buildings, and animated.
Output to a media server or to tape in native formats or MPEG2

The control room can access graphics from tape or as digital files provided by an SGI media server that also stores live footage.

Both Richard O’Brien and Robert Hunter learned valuable lessons in dealing with the insanity of the post attacks period. They are now working to further streamline their operations for the fastest throughput. Teamwork was essential, and both praised their fellow workers who worked long, long hours. Being prepared for the next crisis is a key issue for them. Ultimately, they both seemed to feel that they had produced what was needed to keep their viewers apprised of a dynamic and unprecedented situation. Frankly, I think they did a damn good job.

Rescue

The first wave of emergency personnel at the World Trade Center were largely firefighters, reinforced by City police and Port Authority police. They didn’t rely on high tech, but on guts and strength. Of their number, 343 fire fighters, 23 police officers, and 37 Port Authority police perished as the towers collapsed, after helping to save the lives of tens of thousands of people who had been working in the skyscrapers. (The average daily population of the towers alone was 20,000 employees, plus as many as 10,000 visitors. The most current tally of the dead and missing in New York puts that number at around 2,600, including 403 fire and police personnel, and 157 passengers and crew on both planes.) To give you sense of the magnitude of this loss, in a typical year, fewer than 100 firefighters die on duty in the entire United States.

The second wave of emergency people came in the shadows of the collapses. Very quickly, thermal mapping equipment and LIDAR (Light Detection and Ranging) systems were being used to create detailed maps of the devastated complex. These maps guided rescue personnel around hot spots and other dangerous areas.

Mapping flyovers of the WTC site commenced during the afternoon of September 11, and continued for several weeks. Sean Ahearn, a professor at New York’s Hunter College was among the first to start the mapping of the extent of damage, using LIDAR, also called airborne laser swath mapping (ALSM).

This LIDAR image of the World Trade Center site, taken on September 27, shows the extent of the devastation, as well as many features that would be difficult to see from ground level. Photo courtesy of NOAA-U.S. Army JPSD.

LIDAR works a lot like RADAR, but in a different region of the spectrum, using photons generated by lasers, in place of the radio frequencies RADAR uses. LIDAR is more versatile than RADAR, and can provide rotation, position, and acceleration information, as well as objects outlines quite accurately. For mapping, it is combined with GPS (Global Positioning System) data, and position data obtained from inertial sensor on the aircraft.

To produce scans of the site, Optech, the Canadian manufacturer of LIDAR systems commonly used by the US Government, mounted a laser range finder aimed out of an opening in the floor of a small plane. The range finder scanned beneath the aircraft, measuring the distance to the ground. The width of the scan is a function of altitude and desired accuracy; a swath from 1,000 meter altitude will be 30-68 meters wide. The angle at which the laser is scanned is recorded for each scan. A GPS receiver in the aircraft recorded the aircraft's position at regular intervals while a second, ground-based receiver provided differential correction for a more accurate position estimate. To correct for the aircraft's movements, the motions of the aircraft are recorded by an inertial reference system (IRS). The plane was flown in a raster pattern to provide complete coverage of the site.

In post-flight processing, the laser range, scan angle, GPS and IRS data are combined to determine the absolute position of every point in the scan relative to the Earth's surface. For Ahearn’s WTC site map made on the first day of mapping, the Optech ALTM 2033 laser system was set to provide horizontal accuracy of 45-50 cm (~18"), and the more important vertical accuracy at 9 cm (~3.5").

Dr. Ahearn told me that the LIDAR data was extremely valuable for logistics support. The fire and rescue crews used it extensively every day, in the form of large-scale, printed maps that hung in many locations around the site. The WTC site presented unusual problems, especially for the first response teams. There were holes open to the surface that dropped 60 feet (18 meters) below sea level. The entire plaza, buried in more than 70 feet (21 meters) of debris, had sunk below the original grade level. People were dwarfed in this chaotic site, which dwarfed humans in its scale and extent. Dr. Ahearn explained, "That’s their map. There’s 16 acres of total destruction. There’s not a single thing that’s recognizable to your usual senses. Having something where you could see the shapes and size of the piles was an invaluable reference." He explained that fire fighters used the map to pick out landmarks in this terrain. "Fireman said ‘I love that map. We’re always using it. Whenever we have a question, we go to that map.’"

Like Gulliver in Bromdingnag, rescue workers are dwarfed by the wreckage of the Twin towers. Photo by Michael Rieger- FEMA News Photo

The first map posted at the site was around 50" x 60" and was hung in the Fire Department’s command post at the WTC site. At a scale of around 1:1000, it shows the 16 acre (6.5 hectares) site in a highly detailed rendition that indicates objects as small as 45-50 cm (~18"). This resolution was chosen because it matches the New York City map (NYCMap) that has been under development by the City for five years, and includes every feature of interest to city officials. The high accuracy of the NYCMap, combined with the LIDAR data, pinpointed support structures, elevator shafts, utility connections and basement storage areas and meant that recovery efforts were targeted in the areas where survivors were most likely to be found, and that were accessible within acceptable risk levels.

Monitoring went on for five weeks, until it was no longer needed. By that time most of the fires had subsided, and the dangers of accessing the site were becoming manageable.

At the same time the aerial photos and scans were being completed, the New York City Emergency Mapping Data Center (EMDC) was created to deal with the demand for maps and GIS data for the site. Paul W. Katzer stepped into the Center manager’s job. He told me that the center supplied more than 6,000 maps in the first week after the attack. They used workstations and servers donated by Compaq and a plotter provided by Hewlett-Packard to augment the machines they were able to find in city offices and those brought in by city workers and volunteers. Software included ESRI’s ARCView and ARCInfo, Urban Data Solutions products, and Microsoft Office for reporting and tracking.

Aerial photos such as this one taken by NOAA on September 23 provided the basis for damage assessment and photogrametric maps. Photo courtesy of NOAA.

EMDC quickly developed a tracking application that let them know which maps were requested by whom, and to be ready with the most-requested maps on a few minutes notice. These maps pinpointed not just the topology of the wreckage, but also showed first aid stations, mess halls, veterinarian stations, toilets and other points of importance to the managers and workers on site. One popular map showed safety equipment requirements in various zones of the site, so workers knew exactly what was required to enter any given area.

This site damage map, prepared by NOAA, is similar to the maps used to brief managers and the public. Maps like these were printed by the thousands in the first weeks after the attack. Map courtesy NOAA.

By the second week, EMDC started the Deep Infrastructure project. This project was intended to create 3D maps of all underground structures: freon tanks, fuel storage, elevator shafts, stairways, subway and regional transit tunnels, electrical, steam, gas, and communications lines. This project is highly secure, and will not be made public. However, the plan now is to extend Deep Infrastructure mapping to the entire city of new York. This information is registered to the NYCMap, and will be used for future emergency planning, as well as urban planning.

These four images are all portions of different layers of the NYCMap, all showing lower Manhattan. The layer labels here are arbitrary. A: Layer 1 shows census tracts from the 2000 census. B: Layer 2 shows street center lines. C: Layer 3 indicates the borders of fire districts. D: Layer 4 indicates city health districts. The NYCMap has many more layers, some of which aren’t publicly available because of security issues.

Remember

Less than two weeks after the attack, the New York Times presented a concept rendering of a memorial to the World Trade Center and the victims of the attack. This was one of dozens of ideas for immediate and permanent memorials to the tragedy. The power of computer graphics software was used to make photo-realistic presentations that enable the community to consider the proposals thoroughly.

Several web sites sprung up within hours of the attack, including one developed by Paul Henry Smith and Jarrett Mellenbruch, friends who spent the evening after the attacks watching the news on TV. Smith is a musician by training, but was working a day job as a web designer in mid-town Manhattan on September 11th. Mellenbruch is a visual artist, and they simultaneously arrived at idea to do something as both a reminder of the loss and to convey hopefulness. They started kicking around ideas and converged on the concept reaching up to heaven or some higher good.
The team started with some photos of the Trade Center that Mellenbruch had taken previously. Then, working on a Macintosh in PhotoShop, they rendered an image of the altered lower Manhattan skyline with the diaphanous ghostly outline of the twin towers shaped by light. Interestingly, several others have come up with a similar concepts, including one featured on the cover of the New York Times Sunday Magazine on September 23, 2001. (See it at: http://www.nytimes.com/2001/09/23/magazine/23MYODA.html).

The WTC Beacon is one of many proposed memorials for the victims of the attack in New York. Rendering courtesy of Paul Henry Smith and Jarrett Mellenbruch.

Smith told me that developing the memorial and the web site that publicizes it was a cathartic experience. "Having the ability to put the images together and put them on the internet, really surprised me in that it fulfilled both Jarrett’s and my need to express something. You need a sense that you’ve been hurt and seen. We’re very pleased that we got immediate, and very positive response, from the internet community. In a way, that mitigates the frustration we might have felt if the web were not available to us.

if the web were not available to us.
"Before we did it [the PhotoShop image], we thought we’d do a sketch. But we wanted it to be thought of in the same world as all the live, video, and photo images you were seeing. We wanted it to seem real, that’s why we used PhotoShop. We didn’t want it to look like a sketch."

Many people outside of NY didn’t realize this was just a proposed memorial, but thought it actually existed. Smith and Mellenbruch got several hundred letters thanking then for creating the memorial.
Computer graphics as an emotional release? That’s what we have seen after the September attacks. And while these applications of CG have not particularly novel in themselves, the emotional benefits of them have probably never been seen before.

Rebuild

Of course, whatever becomes of the WTC site, the planning and design will involve computer graphics. But a new level of integration is making it possible to see complex interactions for the first time. Buildings, parks, roads, sidewalks, trains, even bicycle paths, will be individual designed, then assembled into the City of New York’s 3D GIS database that accurately portrays the size and 3D location of sewers, water and gas mains, steam pipe, electrical and communications service lines, subways, roads, sidewalks, bridges, and buildings.

A superset of the GIS database, the NYCMap mentioned earlier, has been in progress for three years. Michael Kwatler, founder of the Environmental Simulation Center in New York, has been working on the project since its inception. He is working on adding intelligence to such maps. His work fits nicely with the work Ahearn is now back to, adding levels of detail, such as all addresses of every building, lot numbers, and utility services. Virtually every city department at one time or another has created it’s own map. Now the city is trying to include all the essential data on in one digital map. This map eventually will combine a normalized description of every building of the city with floor plans, structural details, and possible the intelligence to quickly provide emergency personnel with the essential information they need in any particular situation.

Some of the planning is less rigorous, more fanciful. A recent exhibition at the Max Protetch Gallery in New York showed the ideas and dreams of more than fifty established and emerging architects. Each presented ideas for structures to be erected at the WTC site. Virtually all of these drawings were created using computer graphics – some relying entirely on CG, others adding content to background images of the city skyline and the WTC site. Here too, computer graphics technology is aiding the city by enabling it to start focusing on the renaissance, rather than the horror.

This proposal from Foreign Office Architects includes detailed structural planning and analysis, as well as a beautiful presentation. Image by Foreign Office Architects, courtesy Max Protetch Gallery.

Understand

After any tragedy, it is human nature to try to understand what happened. Analysis of the steel in the fallen towers may yield the secrets to stronger, safer buildings. While it is unlikely that any skyscraper could withstand the sort of attack that hit the world Trade Center, it may be possible to design buildings that perform even better than the Twin Towers did. (Most engineers who have gone on record agree that the Towers’ held up remarkably well, and that the buildings’ design, a tube around a core, was the single factor that held the number of deaths to a relatively low fraction of the total population of the towers and neighborhood.) Professor of structural engineering at the University of California at Berkeley, Abolhassan Astaneh-Asl is one who is searching for those secrets. Along with David McCallen of Lawrence Livermore National Laboratory, he is creating simulations of the progress of the structural damage. Their team is examining steel from the towers to determine how hot it was, and for how long it was heated. They are also testing various other aspects. This data will be used to power a simulation of the attack, which will show the affects of the impact, fire, and progressive failure of the building’s components. This system will give them the ability to vary the materials and building systems to determine optimal designs and materials.

This section of a WTC support column was probably struck by an engine, which took a large chunk out of it. Abolhassan Astaneh-Asl examines it to collect data for the model of the building and incident that he’s creating.

Astaneh-Asl told us that his work is proceeding more slowly than he expected because the Port Authority, owner of the WTC, has not given him the buildings’ plans. These are needed to set up a simulation of what happened. When he does receive those plans, he will work with the computer scientists at Lawrence Livermore National Laboratories to created a detailed simulation that will help understand the damage caused by the attack, and to build structures that can better resist such an assault.

Epilogue

Not long after its completion, Minoru Yamasaki, architect of the World Trade Center, said, 'The World Trade Center should, because of its importance, become a living representation of man's belief in humanity, his need for individual dignity, his belief in the cooperation of men, and through this cooperation his ability to find greatness." Let us hope that whatever the World Trade Center site is ultimately used for, it will be an even more meaningful expression of humanity’s ability to achieve greatness, despite the attempts of small-minded people to keep humanity shackled and ignorant. And let us take pride as graphics professionals in the contributions we have made.

1 Paul Heyer. Architects on Architecture: New Directions in America. Walker & Co., 1978.

Ben Delaney has written many articles for CG&A. He is president of CyberEdge Information Services, Inc., which provides intelligence and analysis of interface technologies, VizSim, and technology’s impact on society. Reach him at ben at cyberedge.com.

Questions? More Info? Email the Webmaster

407 M. L. King Jr. Way, Oakland, CA 94607 USA
+1 510 419-0800

Our privacy policy may be read here.