In IIJ's outside-air-cooled container based data center experiment facility, all 300 servers are running OpenBSD!

I just invited IIJ's next-gen data center experiment facility, and they suggested me to write blog entries.

So I'll try to describe their experiment facilities little bit.

Most important thing for me is it runs OpenBSD anyway ;-)

Problem of traditional data center

Constructing cost is expensive, have to construct all at once.

Takes almost one year to finishing construct, it's difficult to supply to fit demand.

So they need to construct DC for future demand, actually if to do so, it's very difficult to rise ratio of utilization.

While the ratio doesn't raise enough high, their business faces difficult situation.
In data center, they spent same or more amount of electric power for cooling system than IT devices.

These days the devices shrinking but generate more heat, if fill the devices all spaces in rack, it's very difficult to cool them.

So it's waste spaces in rack.

Let's use outside-air-cooled container

Cooling compressor eats most electric power.

They can reduce if we use outside air to cooling the devices.

They also can reduce constructing cost if we use container, and they don't have to construct all at once anymore.

Container can append anytime, and only takes a half day to install.

Experiment

ASHRAE released environmental guidelines(temperature and humidity) for datacenter equipment in 2008, this experiment is trying to follow it using outside air cooling system.
Since they use outside air, temperature and humidity changes by season and also time.
So it needs control.
This is image cooling system, it has three modes:

In winter mode, out side air is too cold, if the system use that air directly, the devices will gets bedewing.

So the system mix exhaust heat and flesh air to cooling.
Spring and fall mode, it just use outside air directly and throw exhaust heat outside.
Summer mode, out side air is too hot, it cannot use it. Use cooling compressor just like traditional datacenter.
For guidelines of power consumption efficiency, we have PUE(Power Usage Effectiveness).

On traditional datacenter, it's generally 2.x, and best score is around 1.4x.

There's no datacenter used as business available under 1.2 in Japan.
This experiment goal is 1.1 whole year for business.

Currently in experiment, they had got 1.07 in winter day(but without UPS and power switchboard).

They are continuously challenging reduce the power consumption to get better score.

To entering business service

They'll construct new datacenter for their cloud service called "IIJ GIO", service in period is next year.

YES IT RUNS OpenBSD!!!!

All servers in the experiment facilities OpenBSD installed!

Thurmo-hygrograph censor is the product of strawberry linux Co., connected to the servers via USB.

And the drivers written by yuo@openbsd. Yeah he also suggested installing OpenBSD to these servers, of cause.

This is great ;-)
These servers are basically just for generating heat(except logging temperature and humidity), it executes openssl to make processors busy.

And of cause it's login from remote, so they can watch temperature log and can control how many machines executes openssl(to heat container up).

Entire picture

Related companies

Cooling system

Cooling compressor

Door for maintenance cooling system

Connector between container and cooling system

Door of container

Foundation of container

It's double entry doors

LED light

Power switch. I'll die if I touched

Thurmo-hygrograph censor

"Hot aisle"

Thurmo-hygrograph censor connected via USB

Duct for throw exhaust heat out

Tube for fire censor

"Cool aisle"

Variety of servers

"It's actually 4GB RAM!" sticker

Tons of 1U servers

Duct for flesh air in

Which manufacturer's machine is this white one?

Other photos #1
Other photos #2