Category Status update

Apparently PG&E is as excited as we are to have RFO back online. 😉 The PSPS outage only lasted a couple of days, but it stings to miss even a single night of observing!

After much work, coordination, and patience, the summit of Haleakala has been reenergized since the fire forced them to cut power. It’s always nice to reconnect and find healthy instruments, ready to do science!

The bad news is we won’t be doing any observing tonight, given the weather. Oh well.

At least it’s a beautiful night at RFO!

We lost power to the summit of Haleakala at approximately 9am Pacific, when Maui Fire Dept requested de-energization to prevent further ignitions and risk to personnel. They’re currently projecting power to remain off through Saturday night.

Thus far, no injuries and structural damages were due to the fire, which is located at around the 7,000-foot elevation level. Four engines, three wildland engines, 11 tankers, five dozers, two helicopters, an Maui Fire Department crew and a 27-person state Department of Land and Natural Resources Division of Forestry and Wildlife’s wildland crew were utilized to battle the fire on Thursday. Crater Road remained closed at last report. The Summit District of Haleakalā National Park remains closed until further notice.

We thank all those involved in protecting life and property, as well as observatory efforts to work with first responders and get us back online as soon as possible.

For more information, please see Maui Now or other news sources.

We lost power at the summit of Haleakala today. Looks like planned electrical work, as it was from roughly 9am to 5pm Hawaiian time. I know MECO/Hawaiian Electric has some infrastructure upgrades to do, so this is a Good Thing if it means we avoid more multi-week outages in the future when big storms hit.

Our UPS (Uninterruptible Power Supply–a fancy battery) kept us running for almost two hours before giving out. That’s slightly under the predicted runtime, but still solid performance I’m happy to see.

Unfortunately, the network upstream from us isn’t battery backed, so there’s nothing we can do remotely when these happen. We just have to let the software execute the automatic failsafe, then check that everything is ok when it comes back online.

In a weather update, this winter has been the worst for observing since we started. Normally winter is one of our best times: a lot of long dark nights with low humidity. This winter has not been that, at either observatory. We did start getting some good–and even simultaneously good–nights in the past couple weeks. Fingers crossed for more clear skies!

Every year, now for the past 3 years, there’s a giant wind/ice storm that knocks power and connectivity out at the summit of Haleakala. This year, it started over a week ago, and we’ve just regained connectivity to the instruments. Hooray!

Fingers crossed we’re done for this year and that the infrastructure improvements they’re doing make next year even shorter or a non-event.

We were actually observing when the power went out, but our multiple levels of safeguards obviously did their job. As you can see, the cameras are in fine working order.

It sure would be nice if the moon weren’t so bright, though! Even outside of the field of view, the two bright semi-circles in the bottom half of the images are scattered light from it.

Making our next 10 instruments efficiently, as well as ensuring their reliability and ease of maintenance, is a top priority. Some of you might remember a previous iteration of a power board, an attempt to reduce wiring and build effort that encompassed all power wiring in the instrument and attempted to eliminate a COTS (commercial off the shelf) component. That board, however, was overpriced and unnecessarily aggressive, so we switched vendors, changed how we did the cabling to avoid joints, and reduced the scope of the board to handling the two remaining wiring challenges: the fans and FLI cameras.

The result is above and below, and is 40x cheaper than before, while still eliminating all of the manual labor, difficulty when swapping out components, and long-term reliability risk! Fans plug into 4-pin headers, and FLI cables will screw into block terminals (not rendered below) in the bottom right. No soldering, everything labelled, quicker and better than could be done by hand.

We hope you enjoy these peeks into the details behind LaserSETI, but stay tuned for some bigger updates and announcements soon!

Since we’re building 10 new instruments, we’ve made a number of small investments to make building them easier and faster. This was a fun one to share:

On the left, you’ll see an off-the-shelf appliance dolly. Cheap and easy to obtain, but it only allows access to the wiring area in the instrument base with the panels removed and with the dolly somewhat awkwardly interfering with said access. On the right, we’ve machined custom aluminum “legs” with casters, so the bottom panels can stay on and even be opened or closed as necessary. The machining took a couple of hours, however, so we’ll be testing both side-by-side, to see which fits the need better–or maybe both are good for slightly different purposes or phases of construction!

For reference on our development process, we’re including XKCD’s study of tool making below (“automation” in programming-speak). 😉

We didn’t get as much time on the sky this week than normal, as we had work going on at both observatories, RFO and IfA. One was planned, the other not so much.

At IfA, our maintenance was unplanned. IFA2’s sunshade stopped working, which is scary because the sunshade protects from the Sun potentially damaging the shutter on the FLI science cameras. You can see the problem below, using our internal “PiCam”: the polycarbonate gear started to slip around the motor shaft.

We recent started a project to detect the the sunshade position using computer vision to catch exactly this sort of problem automatically but, since we currently check every morning, we caught it manually. Fortunately, Team LaserSETI member Doug Hagan who’s local on Maui, was able to make the long drive up Haleakala and close it by hand.

Speaking of cameras, the horz-cam on IFA1 had started misbehaving transiently. We tried remote diagnostics and troubleshooting but eventually we had to admit it wasn’t succeeding. So we’ve sent a new camera (and a new gear) to Doug, and he’ll be replacing both shortly.

Meanwhile, at RFO in California, they’re upgrading the roof surface of the observatory which our instruments sit directly on top of. We worked with observatory team to come up with a plan for them to coat the the whole roof without disturbing the position of the instruments.

If you’ve been reading the news recently, it won’t surprise you to hear that our instruments on the summit of Haleakala, Maui, have been offline for the past couple days.

The good news is they came back online this afternoon, and seem to be in great shape. In fact, I would assess-timate the high winds even did a good job cleaning the windows.

Of course, that’s the easy part of this post. Our knowledge of the enormity of the tragedy from the fires on Maui grows day over day, and wrenches our hearts ever harder. People, businesses, history, nature, all gone and without warning.

Personally, I haven’t heard from everyone I know there, but each one I do hear from is a gift. It reminds me all too much of the firestorms that struck California in recent years–not “just” a fire, but a fast-moving “natural” blowtorch that creates its own weather as it decimates everything in its path. I keep thinking about a sign that hangs in my mother-in-law’s house, and she was kind enough to take a picture of it for me.

If there’s anything you can do to help the wonderful people of Maui recover, now or in the future, I hope you’ll act on that opportunity. The best resource I could find on ways to help was here.

All the life we know of clings to the thin skin of a giant rock flying through space. I think we should take care of each other.

The hard drive in RFO2 has given us trouble before. A year ago, we swapped out the disk, the enclosure, and even the computer itself. For a while, it was smooth sailing, but then started acting up again.

So, after replacing the disk and enclosure again, we tried the power and data cables. That was an improvement, but still not 100%. And since we can’t observe without the hard drive, anything less than perfect isn’t good enough.

We’ll be headed up to Ferguson again soon, this time with a new idea in hand, to try to put this trouble maker to bed once and for all: a solid state disk. Smaller capacity, but completely different performance, electrical, and other characteristics.

Since RFO2 wants to be an attention-hogging snowflake, here’s the work from its perspective. LaserSITTIE is, by far, my favorite piece of maintenance equipment. There’s nothing like sitting when we previously had to bend over the instrument, or kneeling on it instead of the often-sweltering roof.