Data center

From HandWiki
Short description: Building or room used to house computer servers and related equipment
ARSAT data center (2014)

A data center (American English)[1] or data centre (Commonwealth English)[2][note 1] is a building, a dedicated space within a building, or a group of buildings[3] used to house computer systems and associated components, such as telecommunications and storage systems.[4][5]

Since IT operations are crucial for business continuity, it generally includes redundant or backup components and infrastructure for power supply, data communication connections, environmental controls (e.g., air conditioning, fire suppression), and various security devices. A large data center is an industrial-scale operation using as much electricity as a small town.[6] Estimated global data center electricity consumption in 2022 was 240-340 TWh, or roughly 1-1.3% of global electricity demand. This excludes energy used for cryptocurrency mining, which was estimated to be around 110 TWh in 2022, or another 0.4% of global electricity demand.[7]

Data centers can vary widely in terms of size, power requirements, redundancy, and overall structure. Four common categories used to segment types of data centers are onsite data centers, colocation facilities, hyperscale data centers, and edge data centers.[8]

History

NASA mission control computer room c. 1962

Data centers have their roots in the huge computer rooms of the 1940s, typified by ENIAC, one of the earliest examples of a data center.[9][note 2] Early computer systems, complex to operate and maintain, required a special environment in which to operate. Many cables were necessary to connect all the components, and methods to accommodate and organize these were devised such as standard racks to mount equipment, raised floors, and cable trays (installed overhead or under the elevated floor). A single mainframe required a great deal of power and had to be cooled to avoid overheating. Security became important – computers were expensive, and were often used for military purposes.[9][note 3] Basic design guidelines for controlling access to the computer room were therefore devised.

During the boom of the microcomputer industry, and especially during the 1980s, users started to deploy computers everywhere, in many cases with little or no care about operating requirements. However, as information technology (IT) operations started to grow in complexity, organizations grew aware of the need to control IT resources. The availability of inexpensive networking equipment, coupled with new standards for the network structured cabling, made it possible to use a hierarchical design that put the servers in a specific room inside the company. The use of the term data center, as applied to specially designed computer rooms, started to gain popular recognition about this time.[9][note 4]

The boom of data centers came during the dot-com bubble of 1997–2000.[10][note 5] Companies needed fast Internet connectivity and non-stop operation to deploy systems and to establish a presence on the Internet. Installing such equipment was not viable for many smaller companies. Many companies started building very large facilities, called internet data centers (IDCs),[11] which provide enhanced capabilities, such as crossover backup: "If a Bell Atlantic line is cut, we can transfer them to ... to minimize the time of outage."[11]

The term cloud data centers (CDCs) has been used.[12] Data centers typically cost a lot to build and maintain.[10] Increasingly, the division of these terms has almost disappeared and they are being integrated into the term data center.[13]

Requirements for modern data centers

Racks of telecommunications equipment in part of a data center

Modernization and data center transformation enhances performance and energy efficiency.[14]

Information security is also a concern, and for this reason, a data center has to offer a secure environment that minimizes the chances of a security breach. A data center must, therefore, keep high standards for assuring the integrity and functionality of its hosted computer environment.

Industry research company International Data Corporation (IDC) puts the average age of a data center at nine years old.[14] Gartner, another research company, says data centers older than seven years are obsolete.[15] The growth in data (163 zettabytes by 2025[16]) is one factor driving the need for data centers to modernize.

Focus on modernization is not new: concern about obsolete equipment was decried in 2007,[17] and in 2011 Uptime Institute was concerned about the age of the equipment therein.[note 6] By 2018 concern had shifted once again, this time to the age of the staff: "data center staff are aging faster than the equipment."[18]

Meeting standards for data centers

The Telecommunications Industry Association's Telecommunications Infrastructure Standard for Data Centers[19] specifies the minimum requirements for telecommunications infrastructure of data centers and computer rooms including single tenant enterprise data centers and multi-tenant Internet hosting data centers. The topology proposed in this document is intended to be applicable to any size data center.[20]

Telcordia GR-3160, NEBS Requirements for Telecommunications Data Center Equipment and Spaces,[21] provides guidelines for data center spaces within telecommunications networks, and environmental requirements for the equipment intended for installation in those spaces. These criteria were developed jointly by Telcordia and industry representatives. They may be applied to data center spaces housing data processing or Information Technology (IT) equipment. The equipment may be used to:

  • Operate and manage a carrier's telecommunication network
  • Provide data center based applications directly to the carrier's customers
  • Provide hosted applications for a third party to provide services to their customers
  • Provide a combination of these and similar data center applications

Data center transformation

Data center transformation takes a step-by-step approach through integrated projects carried out over time. This differs from a traditional method of data center upgrades that takes a serial and siloed approach.[22] The typical projects within a data center transformation initiative include standardization/consolidation, virtualization, automation and security.

  • Standardization/consolidation: Reducing the number of data centers[23][24] and avoiding server sprawl[25] (both physical and virtual)[26] often includes replacing aging data center equipment,[27] and is aided by standardization.[28]
  • Virtualization: Lowers capital and operational expenses,[29] reduces energy consumption.[30] Virtualized desktops can be hosted in data centers and rented out on a subscription basis.[31] Investment bank Lazard Capital Markets estimated in 2008 that 48 percent of enterprise operations will be virtualized by 2012. Gartner views virtualization as a catalyst for modernization.[32]
  • Automating: Automating tasks such as provisioning, configuration, patching, release management, and compliance is needed, not just when facing fewer skilled IT workers.[28]
  • Securing: Protection of virtual systems is integrated with the existing security of physical infrastructures.[33]

Raised floor

Perforated cooling floor tile
Main page: Engineering:Raised floor

A raised floor standards guide named GR-2930 was developed by Telcordia Technologies, a subsidiary of Ericsson.[34]

Although the first raised floor computer room was made by IBM in 1956,[35] and they've "been around since the 1960s",[36] it was the 1970s that made it more common for computer centers to thereby allow cool air to circulate more efficiently.[37][38]

The first purpose of the raised floor was to allow access for wiring.[35]

Lights out

The lights-out[39] data center, also known as a darkened or a dark data center, is a data center that, ideally, has all but eliminated the need for direct access by personnel, except under extraordinary circumstances. Because of the lack of need for staff to enter the data center, it can be operated without lighting. All of the devices are accessed and managed by remote systems, with automation programs used to perform unattended operations. In addition to the energy savings, reduction in staffing costs and the ability to locate the site further from population centers, implementing a lights-out data center reduces the threat of malicious attacks upon the infrastructure.[40][41]

Noise levels

Generally speaking, local authorities prefer noise levels at data centers to be "10 dB below the existing night-time background noise level at the nearest residence."[42]

OSHA regulations require monitoring of noise levels inside data centers if noise exceeds 85 decibels.[43] The average noise level in server areas of a data center may reach as high as 92-96 dB(A).[44]

Residents living near data centers have described the sound as "a high-pitched whirring noise 24/7", saying “It’s like being on a tarmac with an airplane engine running constantly ... Except that the airplane keeps idling and never leaves.”[45][46][47][48]

External sources of noise include HVAC equipment and energy generators.[49][50]

Data center design

The field of data center design has been growing for decades in various directions, including new construction big and small along with the creative re-use of existing facilities, like abandoned retail space, old salt mines and war-era bunkers.

  • a 65-story data center has already been proposed[51]
  • the number of data centers as of 2016 had grown beyond 3 million USA-wide, and more than triple that number worldwide[10]

Local building codes may govern the minimum ceiling heights and other parameters. Some of the considerations in the design of data centers are:

A typical server rack, commonly seen in colocation
  • Size - one room of a building, one or more floors, or an entire building,
  • Capacity - can hold up to or past 1,000 servers[52]
  • Other considerations - Space, power, cooling, and costs in the data center.[53]
  • Mechanical engineering infrastructure - heating, ventilation and air conditioning (HVAC); humidification and dehumidification equipment; pressurization.[54]
  • Electrical engineering infrastructure design - utility service planning; distribution, switching and bypass from power sources; uninterruptible power source (UPS) systems; and more.[54][55]
CRAC Air Handler

Design criteria and trade-offs

  • Availability expectations: The costs of avoiding downtime should not exceed the cost of the downtime itself[56]
  • Site selection: Location factors include proximity to power grids, telecommunications infrastructure, networking services, transportation lines and emergency services. Other considerations should include flight paths, neighboring power drains, geological risks, and climate (associated with cooling costs).[57]
    • Often, power availability is the hardest to change.

High availability

Main page: High availability

Various metrics exist for measuring the data-availability that results from data-center availability beyond 95% uptime, with the top of the scale counting how many nines can be placed after 99%.[58]

Modularity and flexibility

Modularity and flexibility are key elements in allowing for a data center to grow and change over time. Data center modules are pre-engineered, standardized building blocks that can be easily configured and moved as needed.[59]

A modular data center may consist of data center equipment contained within shipping containers or similar portable containers.[60] Components of the data center can be prefabricated and standardized which facilitates moving if needed.[61]

Environmental control

Temperature and humidity are controlled via:

It is important that computers do not get humid or overheat, as high humidity can lead to dust clogging the fans, which leads to overheat, or can cause components to malfunction, ruining the board and running a fire hazard. Overheat can cause components, usually the silicon or copper of the wires or circuits to melt, causing connections to loosen, causing fire hazards.

Electrical power

A bank of batteries in a large data center, used to provide power until diesel generators can start

Backup power consists of one or more uninterruptible power supplies, battery banks, and/or diesel / gas turbine generators.[64]

To prevent single points of failure, all elements of the electrical systems, including backup systems, are typically given redundant copies, and critical servers are connected to both the A-side and B-side power feeds. This arrangement is often made to achieve N+1 redundancy in the systems. Static transfer switches are sometimes used to ensure instantaneous switchover from one supply to the other in the event of a power failure.

Low-voltage cable routing

Options include:

  • Data cabling can be routed through overhead cable trays[65]
  • Raised floor cabling, both for security reasons and to avoid the extra cost of cooling systems over the racks.
  • Smaller/less expensive data centers may use anti-static tiles instead for a flooring surface.

Air flow

Air flow management addresses the need to improve data center computer cooling efficiency by preventing the recirculation of hot air exhausted from IT equipment and reducing bypass airflow. There are several methods of separating hot and cold airstreams, such as hot/cold aisle containment and in-row cooling units.[66]

Aisle containment

Cold aisle containment is done by exposing the rear of equipment racks, while the fronts of the servers are enclosed with doors and covers. This is similar to how large-scale food companies refrigerate and store their products.

Typical cold aisle configuration with server rack fronts facing each other and cold air distributed through the raised floor

Computer cabinets/Server farms are often organized for containment of hot/cold aisles. Proper air duct placement prevents the cold and hot air from mixing. Rows of cabinets are paired to face each other so that the cool and hot air intakes and exhausts don't mix air, which would severely reduce cooling efficiency.

Alternatively, a range of underfloor panels can create efficient cold air pathways directed to the raised-floor vented tiles. Either the cold aisle or the hot aisle can be contained.[67]

Another option is fitting cabinets with vertical exhaust duct chimneys.[68] Hot exhaust pipes/vents/ducts can direct the air into a Plenum space above a Dropped ceiling and back to the cooling units or to outside vents. With this configuration, traditional hot/cold aisle configuration is not a requirement.[69]

Fire protection

FM200 fire suppression tanks

Data centers feature fire protection systems, including passive and Active Design elements, as well as implementation of fire prevention programs in operations. Smoke detectors are usually installed to provide early warning of a fire at its incipient stage.

Although the main room usually does not allow Wet Pipe-based Systems due to the fragile nature of Circuit-boards, there still exist systems that can be used in the rest of the facility or in cold/hot aisle air circulation systems that are closed systems, such as:[70]

  • Sprinkler systems
  • Misting, using high pressure to create extremely small water droplets, which can be used in sensitive rooms due to the nature of the droplets.

However, there also exist other means to put out fires, especially in Sensitive areas, usually using Gaseous fire suppression, of which Halon gas was the most popular, until the negative effects of producing and using it were discovered.[1]

Security

Main page: Data center security

Physical access is usually restricted. Layered security often starts with fencing, bollards and mantraps.[71] Video camera surveillance and permanent security guards are almost always present if the data center is large or contains sensitive information. Fingerprint recognition mantraps are starting to be commonplace.

Logging access is required by some data protection regulations; some organizations tightly link this to access control systems. Multiple log entries can occur at the main entrance, entrances to internal rooms, and at equipment cabinets. Access control at cabinets can be integrated with intelligent power distribution units, so that locks are networked through the same appliance.[72]

Energy use

Google Data Center, The Dalles, Oregon
Main page: IT energy management

Energy use is a central issue for data centers. Power draw ranges from a few kW for a rack of servers in a closet to several tens of MW for large facilities. Some facilities have power densities more than 100 times that of a typical office building.[73] For higher power density facilities, electricity costs are a dominant operating expense and account for over 10% of the total cost of ownership (TCO) of a data center.[74]

Greenhouse gas emissions

In 2020, data centers (excluding cryptocurrency mining) and data transmission each used about 1% of world electricity.[75] Although some of this electricity was low carbon, the IEA called for more "government and industry efforts on energy efficiency, renewables procurement and RD&D",[75] as some data centers still use electricity generated by fossil fuels.[76] They also said that lifecycle emissions should be considered, that is including embodied emissions, such as in buildings.[75] Data centers are estimated to have been responsible for 0.5% of US greenhouse gas emissions in 2018.[77] Some Chinese companies, such as Tencent, have pledged to be carbon neutral by 2030, while others such as Alibaba have been criticized by Greenpeace for not committing to become carbon neutral.[78]

Energy efficiency and overhead

The most commonly used energy efficiency metric for data centers is power usage effectiveness (PUE), calculated as the ratio of total power entering the data center divided by the power used by IT equipment.

[math]\displaystyle{ \mathrm{PUE} = {\mbox{Total Facility Power} \over \mbox{IT Equipment Power}} }[/math]

PUE measures the percentage of power used by overhead devices (cooling, lighting, etc.). The average USA data center has a PUE of 2.0,[79] meaning two watts of total power (overhead + IT equipment) for every watt delivered to IT equipment. State-of-the-art data centers are estimated to have a PUE of roughly 1.2.[80] Google publishes quarterly efficiency metrics from its data centers in operation.[81]

The U.S. Environmental Protection Agency has an Energy Star rating for standalone or large data centers. To qualify for the ecolabel, a data center must be within the top quartile in energy efficiency of all reported facilities.[82] The Energy Efficiency Improvement Act of 2015 (United States) requires federal facilities — including data centers — to operate more efficiently. California's Title 24 (2014) of the California Code of Regulations mandates that every newly constructed data center must have some form of airflow containment in place to optimize energy efficiency.

The European Union also has a similar initiative: EU Code of Conduct for Data Centres.[83]

Energy use analysis and projects

The focus of measuring and analyzing energy use goes beyond what is used by IT equipment; facility support hardware such as chillers and fans also use energy.[84]

In 2011, server racks in data centers were designed for more than 25 kW and the typical server was estimated to waste about 30% of the electricity it consumed. The energy demand for information storage systems is also rising. A high-availability data center is estimated to have a 1 megawatt (MW) demand and consume $20,000,000 in electricity over its lifetime, with cooling representing 35% to 45% of the data center's total cost of ownership. Calculations show that in two years, the cost of powering and cooling a server could be equal to the cost of purchasing the server hardware.[85] Research in 2018 has shown that a substantial amount of energy could still be conserved by optimizing IT refresh rates and increasing server utilization.[86]

In 2011, Facebook, Rackspace and others founded the Open Compute Project (OCP) to develop and publish open standards for greener data center computing technologies. As part of the project, Facebook published the designs of its server, which it had built for its first dedicated data center in Prineville. Making servers taller left space for more effective heat sinks and enabled the use of fans that moved more air with less energy. By not buying commercial off-the-shelf servers, energy consumption due to unnecessary expansion slots on the motherboard and unneeded components, such as a graphics card, was also saved.[87] In 2016, Google joined the project and published the designs of its 48V DC shallow data center rack. This design had long been part of Google data centers. By eliminating the multiple transformers usually deployed in data centers, Google had achieved a 30% increase in energy efficiency.[88] In 2017, sales for data center hardware built to OCP designs topped $1.2 billion and are expected to reach $6 billion by 2021.[87]

Power and cooling analysis

Data center at CERN (2010)

Power is the largest recurring cost to the user of a data center.[89] Cooling it at or below 70 °F (21 °C) wastes money and energy.[89] Furthermore, overcooling equipment in environments with a high relative humidity can expose equipment to a high amount of moisture that facilitates the growth of salt deposits on conductive filaments in the circuitry.[90]

A power and cooling analysis, also referred to as a thermal assessment, measures the relative temperatures in specific areas as well as the capacity of the cooling systems to handle specific ambient temperatures.[91] A power and cooling analysis can help to identify hot spots, over-cooled areas that can handle greater power use density, the breakpoint of equipment loading, the effectiveness of a raised-floor strategy, and optimal equipment positioning (such as AC units) to balance temperatures across the data center. Power cooling density is a measure of how much square footage the center can cool at maximum capacity.[92] The cooling of data centers is the second largest power consumer after servers. The cooling energy varies from 10% of the total energy consumption in the most efficient data centers and goes up to 45% in standard air-cooled data centers.

Energy efficiency analysis

An energy efficiency analysis measures the energy use of data center IT and facilities equipment. A typical energy efficiency analysis measures factors such as a data center's Power Use Effectiveness (PUE) against industry standards, identifies mechanical and electrical sources of inefficiency, and identifies air-management metrics.[93] However, the limitation of most current metrics and approaches is that they do not include IT in the analysis. Case studies have shown that by addressing energy efficiency holistically in a data center, major efficiencies can be achieved that are not possible otherwise.[94]

Computational Fluid Dynamics (CFD) analysis

Main page: Computational fluid dynamics

This type of analysis uses sophisticated tools and techniques to understand the unique thermal conditions present in each data center—predicting the temperature, airflow, and pressure behavior of a data center to assess performance and energy consumption, using numerical modeling.[95] By predicting the effects of these environmental conditions, CFD analysis of a data center can be used to predict the impact of high-density racks mixed with low-density racks[96] and the onward impact on cooling resources, poor infrastructure management practices, and AC failure or AC shutdown for scheduled maintenance.

Thermal zone mapping

Thermal zone mapping uses sensors and computer modeling to create a three-dimensional image of the hot and cool zones in a data center.[97]

This information can help to identify optimal positioning of data center equipment. For example, critical servers might be placed in a cool zone that is serviced by redundant AC units.

Green data centers

Main page: Green data center
This water-cooled data center in the Port of Strasbourg, France claims the attribute green.

Data centers use a lot of power, consumed by two main usages: The power required to run the actual equipment and then the power required to cool the equipment. Power efficiency reduces the first category.

Cooling cost reduction through natural means includes location decisions: When the focus is avoiding good fiber connectivity, power grid connections, and people concentrations to manage the equipment, a data center can be miles away from the users. Mass data centers like Google or Facebook don't need to be near population centers. Arctic locations that can use outside air, which provides cooling, are becoming more popular.[98]

Renewable electricity sources are another plus. Thus countries with favorable conditions, such as Canada,[99] Finland,[100] Sweden,[101] Norway,[102] and Switzerland[103] are trying to attract cloud computing data centers.

Energy reuse

It is very difficult to reuse the heat which comes from air-cooled data centers. For this reason, data center infrastructures are more often equipped with heat pumps.[104] An alternative to heat pumps is the adoption of liquid cooling throughout a data center. Different liquid cooling techniques are mixed and matched to allow for a fully liquid-cooled infrastructure that captures all heat with water. Different liquid technologies are categorized in 3 main groups, indirect liquid cooling (water-cooled racks), direct liquid cooling (direct-to-chip cooling) and total liquid cooling (complete immersion in liquid, see server immersion cooling). This combination of technologies allows the creation of a thermal cascade as part of temperature chaining scenarios to create high-temperature water outputs from the data center.

Dynamic infrastructure

Main page: Dynamic infrastructure

Dynamic infrastructure[105] provides the ability to intelligently, automatically and securely move workloads within a data center[106] anytime, anywhere, for migrations, provisioning,[107] to enhance performance, or building co-location facilities. It also facilitates performing routine maintenance on either physical or virtual systems all while minimizing interruption. A related concept is Composable Infrastructure, which allows for the dynamic reconfiguration of the available resources to suit needs, only when needed.[108]

Side benefits include

Network infrastructure

An operation engineer overseeing a network operations control room of a data center (2006)
An example of network infrastructure of a data center

Communications in data centers today are most often based on networks running the Internet protocol suite. Data centers contain a set of routers and switches that transport traffic between the servers and to the outside world[110] which are connected according to the data center network architecture. Redundancy of the internet connection is often provided by using two or more upstream service providers (see Multihoming).

Some of the servers at the data center are used for running the basic internet and intranet services needed by internal users in the organization, e.g., e-mail servers, proxy servers, and DNS servers.

Network security elements are also usually deployed: firewalls, VPN gateways, intrusion detection systems, and so on. Also common are monitoring systems for the network and some of the applications. Additional off-site monitoring systems are also typical, in case of a failure of communications inside the data center.

Software/data backup

Non-mutually exclusive options for data backup are:

  • Onsite
  • Offsite

Onsite is traditional,[111] and one of its major advantages is immediate availability.

Offsite backup storage

Main page: Off-site data protection

Data backup techniques include having an encrypted copy of the data offsite. Methods used for transporting data are:[112]

  • Having the customer write the data to a physical medium, such as magnetic tape, and then transporting the tape elsewhere.[113]
  • Directly transferring the data to another site during the backup, using appropriate links.
  • Uploading the data "into the cloud".[114]

Modular data center

A 40-foot Portable Modular Data Center

For quick deployment or disaster recovery, several large hardware vendors have developed mobile/modular solutions that can be installed and made operational in a very short amount of time.

Micro data center

Micro Data Centers (MDCs) are access-level data centers which are smaller in size than traditional data centers but provide the same features.[115] They are typically located near the data source to reduce communication delays, as their small size allows several MDCs to be spread out over a wide area.[116][117] MDCs are well suited to user-facing, front end applications.[118] They are commonly used in edge computing and other areas where low latency data processing is needed.[119]

See also

Notes

  1. See spelling differences.
  2. Old large computer rooms that housed machines like the U.S. Army's ENIAC, which were developed pre-1960 (1945), are now referred to as data centers.
  3. Until the early 1960s, it was primarily the government that used computers, which were large mainframes housed in rooms that today we call data centers.
  4. In the 1990s, network-connected minicomputers (servers) running without input or display devices were housed in the old computer rooms. These new "data centers" or "server rooms" were built within company walls, co-located with low-cost networking equipment.
  5. There was considerable construction of data centers during the early 2000s, in the period of expanding dot-com businesses.
  6. In May 2011, data center research organization Uptime Institute reported that 36 percent of the large companies it surveyed expect to exhaust IT capacity within the next 18 months. James Niccolai. "Data Centers Turn to Outsourcing to Meet Capacity Needs". CIO magazine. http://www.cio.com/article/681897/Data_Centers_Turn_to_Outsourcing_to_Meet_Capacity_Needs. Retrieved 2011-09-09. 
  7. instead of chillers/air conditioners, resulting in energy savings

References

  1. "An Oregon Mill Town Learns to Love Facebook and Apple". The New York Times. March 6, 2018. https://www.nytimes.com/2018/03/06/business/apple-facebook-data-center.html. 
  2. "Google announces London cloud computing data centre". July 13, 2017. https://www.bbc.com/news/technology-40590080. 
  3. "Cloud Computing Brings Sprawling Centers, but Few Jobs". The New York Times. August 27, 2016. https://www.nytimes.com/2016/08/27/technology/cloud-computing-brings-sprawling-centers-but-few-jobs-to-small-towns.html. "data center .. a giant .. facility .. 15 of these buildings, and six more .. under construction" 
  4. "From Manhattan to Montvale". The New York Times. April 20, 1986. https://www.nytimes.com/1986/04/20/realestate/postings-from-manhattan-to-montvale.html. 
  5. Ashlee Vance (December 8, 2008). "Dell Sees Double With Data Center in a Container". https://bits.blogs.nytimes.com/2008/12/08/dell-sees-double-with-data-center-in-a-container/. 
  6. James Glanz (September 22, 2012). "Power, Pollution and the Internet". The New York Times. https://www.nytimes.com/2012/09/23/technology/data-centers-waste-vast-amounts-of-energy-belying-industry-image.html. 
  7. "Data centres & networks" (in en-GB). https://www.iea.org/energy-system/buildings/data-centres-and-data-transmission-networks. 
  8. "Types of Data Centers | How do you Choose the Right Data Center?". https://www.maysteel.com/blog/types-of-data-centers-how-do-you-choose-the-right-data-center/. 
  9. 9.0 9.1 9.2 Angela Bartels (August 31, 2011). "Data Center Evolution: 1960 to 2000". https://blog.rackspace.com/datacenter-evolution-1960-to-2000. 
  10. 10.0 10.1 10.2 Cynthia Harvey (July 10, 2017). "Data Center". Datamation. https://www.datamation.com/data-center/what-is-data-center.html. 
  11. 11.0 11.1 John Holusha (May 14, 2000). "Commercial Property/Engine Room for the Internet; Combining a Data Center With a 'Telco Hotel'". The New York Times. https://www.nytimes.com/2000/05/14/realestate/commercial-property-engine-room-for-internet-combining-data-center-with-telco.html. 
  12. H Yuan. Workload-Aware Request Routing in Cloud Data Center. doi:10.1109/JSEE.2015.00020. 
  13. Quentin Hardy (October 4, 2011). "A Data Center Power Solution". https://bits.blogs.nytimes.com/2011/10/04/a-data-center-power-solution. 
  14. 14.0 14.1 "Mukhar, Nicholas. "HP Updates Data Center Transformation Solutions," August 17, 2011". http://www.mspmentor.net/2011/08/17/hp-updates-data-transformation-solutions/. 
  15. "Sperling, Ed. "Next-Generation Data Centers," Forbes, March 15. 2010". Forbes.com. https://www.forbes.com/2010/03/12/cloud-computing-ibm-technology-cio-network-data-centers.html. Retrieved 2013-08-30. 
  16. "IDC white paper, sponsored by Seagate". https://www.seagate.com/files/www-content/our-story/trends/files/Seagate-WP-DataAge2025-March-2017.pdf. 
  17. "Data centers are aging, unsuited for new technologies". December 10, 2007. https://www.datacenterdynamics.com/news/data-centers-are-aging-unsuited-for-new-technologies-and-businesses-are-slow-to-respond-says-new-ari-survey. 
  18. "Data center staff are aging faster than the equipment". Network World. August 30, 2018. https://www.networkworld.com/article/3301883/data-center/data-center-staff-are-aging-faster-than-the-equipment.html. Retrieved December 21, 2018. 
  19. "TIA-942 Certified Data Centers - Consultants - Auditors - TIA-942.org". http://www.tia-942.org. 
  20. "Telecommunications Standards Development". http://www.tiaonline.org/standards. 
  21. "GR-3160 - Telecommunications Data Center - Telcordia". https://telecom-info.njdepot.ericsson.net/site-cgi/ido/docs.cgi?ID=SEARCH&DOCUMENT=GR-3160&. 
  22. "Tang, Helen. "Three Signs it's time to transform your data center," August 3, 2010, Data Center Knowledge". http://www.datacenterknowledge.com/archives/2010/08/03/three-signs-it%E2%80%99s-time-to-transform-your-data-center/. 
  23. "the Era of Great Data Center Consolidation". Fortune. February 16, 2017. http://fortune.com/2017/02/15/data-center-consolidation-cloud. "'Friends don't let friends build data centers,' said Charles Phillips, chief executive officer of Infor, a business software maker". 
  24. "This Wave of Data Center Consolidation is Different from the First One". February 8, 2018. https://www.datacenterknowledge.com/manage/wave-data-center-consolidation-different-first-one. 
  25. "Start A Fire". http://startafire.com/. 
  26. "Stop Virtual Server Sprawl". http://ibmsystemsmag.com/power/systems-management/virtualization/stop-server-sprawl. 
  27. "Top reasons to upgrade vintage data centers". http://www.eaton.com/content/dam/eaton/company/sustainability/files/thought-leadership-white-paper.pdf. 
  28. 28.0 28.1 "Complexity: Growing Data Center Challenge". May 16, 2007. https://www.datacenterknowledge.com/archives/2007/05/16/complexity-growing-data-center-challenge. 
  29. "Carousel's Expert Walks Through Major Benefits of Virtualization". http://technews.tmcnet.com/channels/virtualization/topics/virtualization/articles/193652-carousels-expert-walks-through-major-benefits-virtualization.htm. 
  30. Stephen Delahunty (August 15, 2011). "The New urgency for Server Virtualization". InformationWeek. http://www.informationweek.com/news/government/enterprise-architecture/231300585. 
  31. "HVD: the cloud's silver lining". Intrinsic Technology. http://www.intrinsictechnology.co.uk/FileUploads/HVD_Whitepaper.pdf. 
  32. "Gartner: Virtualization Disrupts Server Vendors". December 2, 2008. http://www.datacenterknowledge.com/archives/2008/12/02/gartner-virtualization-disrupts-server-vendors/. 
  33. "Ritter, Ted. Nemertes Research, "Securing the Data-Center Transformation Aligning Security and Data-Center Dynamics"". http://lippisreport.com/2011/05/securing-the-data-center-transformation-aligning-security-and-data-center-dynamics/. 
  34. "GR-2930 - NEBS: Raised Floor Requirements". https://telecom-info.njdepot.ericsson.net/site-cgi/ido/docs.cgi?ID=SEARCH&DOCUMENT=GR-2930&. 
  35. 35.0 35.1 "Data Center Raised Floor History". http://ed-thelen.org/comp-hist/RaisedFloor-or-Non-Raised-Floor.pdf. 
  36. "Raised Floor Info | Tips for Ordering Replacement Raised Floor Tiles". https://www.accessfloorsystems.com/index.php/raised-floor-info/tips-for-ordering-replacement-raised-floor-tiles.html. 
  37. Hwaiyu Geng (2014). Data Center Handbook. John Wiley & Sons. ISBN 978-1118436639. https://books.google.com/books?isbn=1118436636. 
  38. Steven Spinazzola (2005). "HVAC: The Challenge And Benefits of Under Floor Air Distribution Systems". http://www.facilitiesnet.com/hvac/article/HVAC-The-Challenges-and-Benefits-of-Under-Floor-Air-Distribution-Systems--3516. 
  39. "Premier 100 Q&A: HP's CIO sees 'lights-out' data centers". Informationweek. March 6, 2006. https://www.computerworld.com/article/2562532/premier-100-q-a--hp-s-cio-sees--lights-out--data-centers.html. 
  40. Victor Kasacavage (2002). Complete book of remote access: connectivity and security. The Auerbach Best Practices Series. CRC Press. p. 227. ISBN 0-8493-1253-1. 
  41. Roxanne E. Burkey; Charles V. Breakfield (2000). Designing a total data solution: technology, implementation and deployment. Auerbach Best Practices. CRC Press. p. 24. ISBN 0-8493-0893-3. 
  42. Clarke, Renaud (2020-07-01). "Acoustic Barriers for Data Centres" (in en-US). https://www.iacacoustics.global/news/acoustic-barriers-for-data-centres/. 
  43. Thibodeau, Patrick (2007-07-31). "That sound you hear? The next data center problem" (in en). https://www.computerworld.com/article/2542868/that-sound-you-hear--the-next-data-center-problem.html. 
  44. Sensear. "Data Center Noise Levels" (in en-us). https://www.sensear.com/markets/data-centers/noise-levels-infographic. 
  45. Weisbrod, Katelyn (2023-02-10). "In Northern Virginia, a Coming Data Center Boom Sounds a Community Alarm" (in en-US). https://insideclimatenews.org/news/10022023/virginia-data-centers-amazon-prince-william-county/. 
  46. Judge, Peter (2022-07-19). "Prince William residents complain of "catastrophic noise" from data centers" (in en). DCD. https://www.datacenterdynamics.com/en/news/prince-william-residents-complain-of-catastrophic-noise-from-data-centers/. 
  47. Judge, Peter (2022-07-27). "Chicago residents complain of noise from Digital Realty data center" (in en). DCD. https://www.datacenterdynamics.com/en/news/chicago-residents-complain-of-noise-from-digital-realty-data-center/. 
  48. Phillips, Mark (2021-11-30). "Chandler to consider banning data centers amid noise complaints" (in en). https://www.abc15.com/news/region-southeast-valley/chandler/chandler-to-consider-banning-data-centers. 
  49. "Data Center Soundproofing and Noise Control- Reduce Server Noise". https://ddsacoustical.com/applications-dds/data-center-soundproofing/. 
  50. Bosker, Bianca (2019-12-06). "Your "cloud" data is making noise on the ground" (in en-US). https://www.marketplace.org/shows/marketplace-tech/your-cloud-data-is-making-noise-on-the-ground/. 
  51. Patrick Thibodeau (April 12, 2016). "Envisioning a 65-story data center". Computerworld. https://www.computerworld.com/article/3054603/data-center/a-65-story-data-center-design-that-soars-with-ideas.html. 
  52. "Google container data center tour (video)". https://www.youtube.com/watch?v=zRwPSFpLX8I. 
  53. "Romonet Offers Predictive Modeling Tool For Data Center Planning". June 29, 2011. http://www.networkcomputing.com/data-center/231000669. 
  54. 54.0 54.1 "BICSI News Magazine - May/June 2010". http://www.nxtbook.com/nxtbooks/bicsi/news_20100506/#/26. 
  55. "Hedging Your Data Center Power". http://www.datacenterjournal.com/design/hedging-your-data-center-power/. 
  56. Clark, Jeffrey. "The Price of Data Center Availability—How much availability do you need?", Oct. 12, 2011, The Data Center Journal "Data Center Outsourcing in India projected to grow according to Gartner". http://www.datacenterjournal.com/home/news/languages/item/2792-the-price-of-data-center-availability. 
  57. "Five tips on selecting a data center location". http://searchcio.techtarget.com/news/1312614/Five-tips-on-selecting-a-data-center-location. 
  58. "IBM zEnterprise EC12 Business Value Video". https://www.youtube.com/watch?v=DPcM5UePTY0. 
  59. Niles, Susan. "Standardization and Modularity in Data Center Physical Infrastructure," 2011, Schneider Electric, page 4. "Standardization and Modularity in Data Center Physical Infrastructure". http://www.apcmedia.com/salestools/VAVR-626VPD_R1_EN.pdf. 
  60. "Strategies for the Containerized Data Center". September 8, 2011. http://www.datacenterknowledge.com/archives/2011/09/08/strategies-for-the-containerized-data-center/. 
  61. Niccolai, James (2010-07-27). "HP says prefab data center cuts costs in half". http://www.infoworld.com/d/green-it/hp-says-prefab-data-center-cuts-costs-in-half-837?page=0,0. 
  62. "tw telecom and NYSERDA Announce Co-location Expansion". Reuters. 2009-09-14. https://www.reuters.com/article/pressRelease/idUS141369+14-Sep-2009+PRN20090914. 
  63. "Air to air combat - indirect air cooling wars". http://www.datacenterdynamics.com/focus/archive/2013/09/air-air-combat-indirect-air-cooling-wars-0. 
  64. Detailed explanation of UPS topologies "EVALUATING THE ECONOMIC IMPACT OF UPS TECHNOLOGY". http://www.emersonnetworkpower.com/en-US/Brands/Liebert/Documents/White%20Papers/Evaluating%20the%20Economic%20Impact%20of%20UPS%20Technology.pdf. 
  65. "Cable tray systems support cables' journey through the data center". April 2016. https://www.cablinginstall.com/articles/print/volume-24/issue-4/features/data-center/cable-tray-systems-support-cables-journey-through-the-data-center.html. 
  66. Mike Fox (2012-02-15). "Stulz announced it has begun manufacturing In Row server cooling units under the name "CyberRow".". DataCenterFix. http://datacenterfix.com/news/stulz-release-cyberrow-row-cooling-units. 
  67. Hot-Aisle vs. Cold-Aisle Containment for Data Centers, John Niemann, Kevin Brown, and Victor Avelar, APC by Schneider Electric White Paper 135, Revision 1
  68. "US Patent Application for DUCTED EXHAUST EQUIPMENT ENCLOSURE Patent Application (Application #20180042143 issued February 8, 2018) - Justia Patents Search" (in en). https://patents.justia.com/patent/20180042143. 
  69. "Airflow Management Basics – Comparing Containment Systems • Data Center Frontier". Data Center Frontier. 2017-07-27. https://datacenterfrontier.com/airflow-management-basics-comparing-containment-systems. 
  70. "Data Center Fire Suppression Systems: What Facility Managers Should Consider". https://www.facilitiesnet.com/datacenters/article/Data-Center-Fire-Suppression-Systems-What-Facility-Managers-Should-Consider--14595. 
  71. Sarah D. Scalet (2005-11-01). "19 Ways to Build Physical Security Into a Data Center". Csoonline.com. http://www.csoonline.com/article/220665. 
  72. Systems and methods for controlling an electronic lock for a remote device, 2016-08-01, https://patents.google.com/patent/US9865109B2/en, retrieved 2018-04-25 
  73. "Data Center Energy Consumption Trends". U.S. Department of Energy. http://www1.eere.energy.gov/femp/program/dc_energy_consumption.html. 
  74. J. Koomey, C. Belady, M. Patterson, A. Santos, K.D. Lange: Assessing Trends Over Time in Performance, Costs, and Energy Use for Servers Released on the web August 17th, 2009.
  75. 75.0 75.1 75.2 "Data Centres and Data Transmission Networks – Analysis" (in en-GB). https://www.iea.org/reports/data-centres-and-data-transmission-networks. 
  76. Kantor, Alice (2021-05-18). "Big Tech races to clean up act as cloud energy use grows". Financial Times. https://www.ft.com/content/c719f655-149c-4ce0-a7a5-18527c7776cf. 
  77. Siddik, Md Abu Bakar; Shehabi, Arman; Marston, Landon (2021-05-21). "The environmental footprint of data centers in the United States" (in en). Environmental Research Letters 16 (6): 064017. doi:10.1088/1748-9326/abfba1. ISSN 1748-9326. Bibcode2021ERL....16f4017S. 
  78. James, Greg (2022-03-01). "Tencent pledges to achieve carbon neutrality by 2030" (in en-US). https://supchina.com/2022/03/01/tencent-pledges-to-achieve-carbon-neutrality-by-2030/. 
  79. "Report to Congress on Server and Data Center Energy Efficiency". U.S. Environmental Protection Agency ENERGY STAR Program. http://www.energystar.gov/ia/partners/prod_development/downloads/EPA_Datacenter_Report_Congress_Final1.pdf. 
  80. "Data Center Energy Forecast". Silicon Valley Leadership Group. https://microsite.accenture.com/svlgreport/Documents/pdf/SVLG_Report.pdf. 
  81. "Efficiency: How we do it – Data centers". https://www.google.com/about/datacenters/efficiency/internal/. 
  82. Commentary on introduction of Energy Star for Data Centers "Introducing EPA ENERGY STAR for Data Centers" (Web site). Jack Pouchet. 2010-09-27. http://www.emerson.com/edc/post/2010/06/15/Introducing-EPA-ENERGY-STARc2ae-for-Data-Centers.aspx. 
  83. "EU Code of Conduct for Data Centres". iet.jrc.ec.europa.eu. http://iet.jrc.ec.europa.eu/energyefficiency/ict-codes-conduct/data-centres-energy-efficiency. 
  84. "UNICOM Global :: Home". http://www.gtsi.com/cms/documents/white-papers/green-it.pdf. 
  85. Daniel Minoli (2011). Designing Green Networks and Network Operations: Saving Run-the-Engine Costs. CRC Press. pp. 5. ISBN 9781439816394. 
  86. Rabih Bashroush (2018). "A Comprehensive Reasoning Framework for Hardware Refresh in Data Centres". IEEE Transactions on Sustainable Computing 3 (4): 209–220. doi:10.1109/TSUSC.2018.2795465. 
  87. 87.0 87.1 Peter Sayer (March 28, 2018). "What is the Open Compute Project?". NetworkWorld. https://www.networkworld.com/article/3266293/lan-wan/what-is-the-open-compute-project.html. 
  88. Peter Judge (March 9, 2016). "OCP Summit: Google joins and shares 48V tech". DCD Data center Dynamics. https://www.datacenterdynamics.com/news/ocp-summit-google-joins-and-shares-48v-tech/. 
  89. 89.0 89.1 Joe Cosmano (2009), Choosing a Data Center, Disaster Recovery Journal, http://www.atlantic.net/images/pdf/choosing_a_data_center.pdf, retrieved 2012-07-21 [yes|permanent dead link|dead link}}]
  90. David Garrett (2004), Heat Of The Moment, Processor, http://www.processor.com/editorial/article.asp?article=articles%2Fp2628%2F21p28%2F21p28.asp, retrieved 2012-07-21 
  91. "HP's Green Data Center Portfolio Keeps Growing - InternetNews.". 25 July 2007. http://www.internetnews.com/xSP/article.php/3690651/HPs+Green+Data+Center+Portfolio+Keeps+Growing.htm. 
  92. Inc. staff (2010), How to Choose a Data Center, http://www.inc.com/guides/2010/11/how-to-choose-a-data-center_pagen_2.html, retrieved 2012-07-21 
  93. "Siranosian, Kathryn. "HP Shows Companies How to Integrate Energy Management and Carbon Reduction," TriplePundit, April 5, 2011.". http://www.triplepundit.com/2011/04/hp-launches-program-companies-integrate-manage-energy-carbon-reduction-strategies. 
  94. Rabih Bashroush; Eoin Woods (2017). "Architectural Principles for Energy-Aware Internet-Scale Applications". IEEE Software 34 (3): 14–17. doi:10.1109/MS.2017.60. 
  95. Bullock, Michael. "Computation Fluid Dynamics - Hot topic at Data Center World," Transitional Data Services, March 18, 2010.
  96. "Bouley, Dennis (editor). "Impact of Virtualization on Data Center Physical Infrastructure," The Green grid, 2010.". http://www.thegreengrid.org/~/media/WhitePapers/White_Paper_27_Impact_of_Virtualization_Data_On_Center_Physical_Infrastructure_020210.pdf?lang=en. 
  97. "HP Thermal Zone Mapping plots data center hot spots". http://searchdatacenter.techtarget.com/news/1265634/HP-Thermal-Zone-Mapping-plots-data-center-hot-spots. 
  98. "Fjord-cooled DC in Norway claims to be greenest". 23 December 2011. http://www.gizmag.com/fjord-cooled-data-center/20938/. 
  99. Canada Called Prime Real Estate for Massive Data Computers - Globe & Mail Retrieved June 29, 2011.
  100. Finland - First Choice for Siting Your Cloud Computing Data Center.. Retrieved 4 August 2010.
  101. "Stockholm sets sights on data center customers". http://www.stockholmbusinessregion.se/templates/page____41724.aspx?epslanguage=EN. 
  102. In a world of rapidly increasing carbon emissions from the ICT industry, Norway offers a sustainable solution Retrieved 1 March 2016.
  103. Swiss Carbon-Neutral Servers Hit the Cloud.. Retrieved 4 August 2010.
  104. "Data Center Cooling with Heat Recovery". January 23, 2017. https://stockholmdataparks.com/wp-content/uploads/white-paper-cost-efficient-and-sustainable-data-center-cooling-2017-01-23-1.pdf. 
  105. "Method for Dynamic Information Technology Infrastructure Provisioning". https://www.freepatentsonline.com/y2007/0294736.html. 
  106. Meyler, Kerrie (April 29, 2008). "The Dynamic Datacenter". https://www.networkworld.com/article/2343998/the-dynamic-datacenter.html. 
  107. "Computation on Demand: The Promise of Dynamic Provisioning". http://www.on-demandenterprise.com/features/26054149.html. [yes|permanent dead link|dead link}}]
  108. "Just What the Heck Is Composable Infrastructure, Anyway?". July 14, 2016. https://www.itprotoday.com/business-resources/just-what-heck-composable-infrastructure-anyway. 
  109. Montazerolghaem, Ahmadreza (2020-07-13). "Software-defined load-balanced data center: design, implementation and performance analysis". Cluster Computing 24 (2): 591–610. doi:10.1007/s10586-020-03134-x. ISSN 1386-7857. https://hal.archives-ouvertes.fr/hal-03048010/file/elsarticle-template.pdf. 
  110. Mohammad Noormohammadpour; Cauligi Raghavendra (July 16, 2018). "Datacenter Traffic Control: Understanding Techniques and Tradeoffs". IEEE Communications Surveys & Tutorials 20 (2): 1492–1525. doi:10.1109/comst.2017.2782753. 
  111. "Protecting Data Without Blowing The Budget, Part 1: Onsite Backup". Forbes. October 4, 2018. https://www.forbes.com/sites/forbestechcouncil/2018/10/04/protecting-data-without-blowing-the-budget-part-1-onsite-backup. 
  112. "Iron Mountain vs Amazon Glacier: Total Cost Analysis". http://spectralogic.com/wp-content/uploads/white-paper-iron-mountain-vs-amazon-glacier.pdf. 
  113. What IBM calls "PTAM: Pickup Truck Access Method." "PTAM - Pickup Truck Access Method (disaster recovery slang)". https://www.acronymfinder.com/Pickup-Truck-Access-Method-(disaster-recovery-slang)-(PTAM).html. 
  114. Iron Mountain introduces cloud backup and management service. September 14, 2017. https://www.networkworld.com/article/3224892/backup-recovery/iron-mountain-introduces-cloud-backup-and-management-service.html. Retrieved October 28, 2018. 
  115. Ibrahim, Rosdiazli; Porkumaran, K.; Kannan, Ramani; Nor, Nursyarizal Mohd; Prabakar, S. (2022-11-13) (in en). International Conference on Artificial Intelligence for Smart Community: AISC 2020, 17–18 December, Universiti Teknologi Petronas, Malaysia. Springer Nature. pp. 461. ISBN 978-981-16-2183-3. https://books.google.com/books?id=6tabEAAAQBAJ&dq=micro+data+center&pg=PA461. 
  116. Guo, Song; Qu, Zhihao (2022-02-10) (in en). Edge Learning for Distributed Big Data Analytics: Theory, Algorithms, and System Design. Cambridge University Press. pp. 12–13. ISBN 978-1-108-83237-3. https://books.google.com/books?id=9IxVEAAAQBAJ&dq=micro+data+center&pg=PA12. 
  117. Resources, Management Association, Information (2022-04-01) (in en). Research Anthology on Edge Computing Protocols, Applications, and Integration. IGI Global. pp. 55. ISBN 978-1-6684-5701-6. https://books.google.com/books?id=hWV2EAAAQBAJ&dq=micro+data+center&pg=PA55. 
  118. Furht, Borko; Escalante, Armando (2011-12-09) (in en). Handbook of Data Intensive Computing. Springer Science & Business Media. pp. 17. ISBN 978-1-4614-1414-8. https://books.google.com/books?id=29UIY9EZTzYC&dq=micro+data+center&pg=PA17. 
  119. Srivastava, Gautam; Ghosh, Uttam; Lin, Jerry Chun-Wei (2023-06-24) (in en). Security and Risk Analysis for Intelligent Edge Computing. Springer Nature. pp. 17. ISBN 978-3-031-28150-1. https://books.google.com/books?id=ahvHEAAAQBAJ&dq=micro+data+center&pg=PA17. 

External links