Category IfA

Every year, now for the past 3 years, there’s a giant wind/ice storm that knocks power and connectivity out at the summit of Haleakala. This year, it started over a week ago, and we’ve just regained connectivity to the instruments. Hooray!

Fingers crossed we’re done for this year and that the infrastructure improvements they’re doing make next year even shorter or a non-event.

We were actually observing when the power went out, but our multiple levels of safeguards obviously did their job. As you can see, the cameras are in fine working order.

It sure would be nice if the moon weren’t so bright, though! Even outside of the field of view, the two bright semi-circles in the bottom half of the images are scattered light from it.

We didn’t get as much time on the sky this week than normal, as we had work going on at both observatories, RFO and IfA. One was planned, the other not so much.

At IfA, our maintenance was unplanned. IFA2’s sunshade stopped working, which is scary because the sunshade protects from the Sun potentially damaging the shutter on the FLI science cameras. You can see the problem below, using our internal “PiCam”: the polycarbonate gear started to slip around the motor shaft.

We recent started a project to detect the the sunshade position using computer vision to catch exactly this sort of problem automatically but, since we currently check every morning, we caught it manually. Fortunately, Team LaserSETI member Doug Hagan who’s local on Maui, was able to make the long drive up Haleakala and close it by hand.

Speaking of cameras, the horz-cam on IFA1 had started misbehaving transiently. We tried remote diagnostics and troubleshooting but eventually we had to admit it wasn’t succeeding. So we’ve sent a new camera (and a new gear) to Doug, and he’ll be replacing both shortly.

Meanwhile, at RFO in California, they’re upgrading the roof surface of the observatory which our instruments sit directly on top of. We worked with observatory team to come up with a plan for them to coat the the whole roof without disturbing the position of the instruments.

If you’ve been reading the news recently, it won’t surprise you to hear that our instruments on the summit of Haleakala, Maui, have been offline for the past couple days.

The good news is they came back online this afternoon, and seem to be in great shape. In fact, I would assess-timate the high winds even did a good job cleaning the windows.

Of course, that’s the easy part of this post. Our knowledge of the enormity of the tragedy from the fires on Maui grows day over day, and wrenches our hearts ever harder. People, businesses, history, nature, all gone and without warning.

Personally, I haven’t heard from everyone I know there, but each one I do hear from is a gift. It reminds me all too much of the firestorms that struck California in recent years–not “just” a fire, but a fast-moving “natural” blowtorch that creates its own weather as it decimates everything in its path. I keep thinking about a sign that hangs in my mother-in-law’s house, and she was kind enough to take a picture of it for me.

If there’s anything you can do to help the wonderful people of Maui recover, now or in the future, I hope you’ll act on that opportunity. The best resource I could find on ways to help was here.

All the life we know of clings to the thin skin of a giant rock flying through space. I think we should take care of each other.

It seems things are slowly returning to normal on Haleakala. The power has been more stable <knocks on wood> and this is the second night in a row of those gorgeous clear skies. We’ve got a lot of observing time to make up!

We’ve also got an exciting announcement coming up about data from Maui. More on that soon!

No less than 75 days after a large storm hit Maui, we heard from the instruments on Haleakala today!

The storm knocked out both power and fiber to the summit and it’s been a long and bumpy road getting power and connectivity back. The turmoil caused all sorts of secondary damage: breakers, generators, routers–all had to be replaced as problems were discovered sequentially. Sometimes temporary repairs had to be undone and redone. Sometimes replaced components had to be replaced again. We’re very grateful to the tenacity of the IfA team, as well as the workers at MECO and Hawaiian Telecom.

As you can perhaps read from the graphs, tonight the humidity was 100%, a strong indication you won’t be seeing any stars. But, as a system test, we observed anyways for a short period and–happily–everything performed as expected. Unfortunately, utility power was lost again, near 10pm local time.

We were only online for a handful of hours, but it’s so nice to know that, after a 2.5 month sabbatical and torrential ice storm, everything is still in great shape! Fingers crossed for more stable power from here on out, and getting back to the science!

We lost power at RFO from about 8am to 5pm today. Everything came back online as expected, however, and we’re observing now.

Not a packet yet from IfA. They found the problem, though: the network was damaged in our building. Physical repair completed, and (eagerly) awaiting reconfiguration to bring us back online.

It’s the best night we’ve had at RFO in at least a month. Lots of stars, clear, not too much moisture, few to no clouds. 🔭 Hooray!

At IfA, however, the saga continues. Power had been “restored” via generator, then the main network cable was fixed… and re-fixed. Then we lost power from the generator, had to replace a major breaker after ordering a replacement from off the island, and now mains power and generator are both up.

As you might have concluded by now, there was a lot of damage to the equipment on the mountain from the storm in December. It’s not clear what was from direct storm damage, power fluctuations, or other secondary causes, but clearly there’s still more to repair because… we still haven’t been able to reach the instruments, despite general connectivity being restored for a third time this month.

My hope is and endeavor will be that this is the last post about what’s not working, and the next one has only good news. Next week? Fingers crossed. Stay tuned and thanks for thinking good thoughts for us!

Just because you’re in Hawaii, doesn’t mean it’s warm! We got our first frost of the season, after being socked in and unable to observe last night.

We can see effects of cooler temperatures in California, too. One of the camera’s in RFO2 is especially prone to being sticky when cold, but fortunately we now have software to handle it automatically.

Ideally, and with most of the other cameras, it takes only 3 attempts to open the shutter all the way 3 times in a row, but last night it took 9.

For the first time in years, I took a trip to Maui where the primary reason was actually to have fun. But of course I wasn’t going to miss the opportunity to visit LaserSETI! And my father-in-law was kind enough to come along for the ride and even give me a hand–literally and figuratively.

It’s always a pleasure, after long drive up the mountain as the last step of the journey, to climb the stairs of the Airglow Building and see the instruments, peacefully sitting amongst the incredible scenery.

The main bit of work we did, besides inspection and maintenance check, was to install a 19V step-up transformer, because we’ve found that the USB bus on this newer generation of NUC computers can be a little finnicky when supplied with 12V like everything else in the instrument. Since the science cameras and disk are connected via USB, any issues in that department are big issues.

Checking in on the air filters, it was clearly time to replace them. Apparently summer is a dusty time up there on the summit. These MERV 13 filters aren’t cheap, but they do a their job very well and have demonstrated a very reasonable operational life expectancy.

As predicted by the forecast below, RFO is socked in tonight. Tomorrow night should be really good, however.

The first three rows mostly characterize moisture in the air (a lot tonight, very little tomorrow night). Seeing represents turbulence, but that doesn’t affect us much because our pixels are big, which is because our field of view (FOV) is enormous (75 degrees!). Darkness matters a lot however for the same FOV reason. And this week, the Moon rises and sets with the Sun, which makes the nights nice and dark because the Moon isn’t lighting up every tiny water droplet and mote of dust that happens to be floating around the sky.

And so this is what the sky currently looks like at Ferguson:

However, at the same time, looking at the exact same stars–but from underneath a different part of the atmosphere–IFA2 is having a great night:

Hopefully, tomorrow night it’ll stay this way in Hawaii but also look like this in California!