Category Status update

Many of LaserSETI’s parts are challenging to print, simply due to the required function of the part and the limitations of Fused Deposition Manufacturing (FDM), aka 3D printing. We’d worked out most of the kinks in the original design of the parts, but one has been giving us trouble.

This is the bracket that holds the “horz-cam” but also, underneath in the bay visible above, holds the router that connects all the various computers that make LaserSETI operate and analyze.

The near side is where all the ethernet cables plug in, so obviously can’t be a solid wall. The highlighted support pillar is large enough to grow into the camera bracket higher up, but small enough to not cause a problem with the cables below. But sometimes, if everything below didn’t print 1000% perfectly, it became a long lever arm that got pushed around by the print head and eventually broke off.

It’s a large part, taking 75 hours of machine time and about $15 in raw plastic. So a print failure more than halfway through is a frustrating waste of time, money, and effort. And it’d happened one time too many times. There are a plethora of factors, mainly the printer and type of thermoplastic, but many variables aren’t realistic to control precisely, such condition of the machine, ambient temperature and humidity, feedstock humidity and other imperfections.

We’d tried fixing it “the right way” by printing a support pillar on the side, meant to be broken off once the print job finishes. But it wasn’t getting a solid enough connection to the model and too often failed in its one job.

As the saying goes, “if you want something done right, do it yourself.” So we carefully crafted a brace from scrap wood, it fits tightly, and we’re expecting this will get the job success rate up, from maybe 65% to more like 100%!

It seems things are slowly returning to normal on Haleakala. The power has been more stable <knocks on wood> and this is the second night in a row of those gorgeous clear skies. We’ve got a lot of observing time to make up!

We’ve also got an exciting announcement coming up about data from Maui. More on that soon!

No less than 75 days after a large storm hit Maui, we heard from the instruments on Haleakala today!

The storm knocked out both power and fiber to the summit and it’s been a long and bumpy road getting power and connectivity back. The turmoil caused all sorts of secondary damage: breakers, generators, routers–all had to be replaced as problems were discovered sequentially. Sometimes temporary repairs had to be undone and redone. Sometimes replaced components had to be replaced again. We’re very grateful to the tenacity of the IfA team, as well as the workers at MECO and Hawaiian Telecom.

As you can perhaps read from the graphs, tonight the humidity was 100%, a strong indication you won’t be seeing any stars. But, as a system test, we observed anyways for a short period and–happily–everything performed as expected. Unfortunately, utility power was lost again, near 10pm local time.

We were only online for a handful of hours, but it’s so nice to know that, after a 2.5 month sabbatical and torrential ice storm, everything is still in great shape! Fingers crossed for more stable power from here on out, and getting back to the science!

We lost power at RFO from about 8am to 5pm today. Everything came back online as expected, however, and we’re observing now.

Not a packet yet from IfA. They found the problem, though: the network was damaged in our building. Physical repair completed, and (eagerly) awaiting reconfiguration to bring us back online.

It’s the best night we’ve had at RFO in at least a month. Lots of stars, clear, not too much moisture, few to no clouds. 🔭 Hooray!

At IfA, however, the saga continues. Power had been “restored” via generator, then the main network cable was fixed… and re-fixed. Then we lost power from the generator, had to replace a major breaker after ordering a replacement from off the island, and now mains power and generator are both up.

As you might have concluded by now, there was a lot of damage to the equipment on the mountain from the storm in December. It’s not clear what was from direct storm damage, power fluctuations, or other secondary causes, but clearly there’s still more to repair because… we still haven’t been able to reach the instruments, despite general connectivity being restored for a third time this month.

My hope is and endeavor will be that this is the last post about what’s not working, and the next one has only good news. Next week? Fingers crossed. Stay tuned and thanks for thinking good thoughts for us!

In the years that LaserSETI’s been operational across two sites, this is certainly the longest weather delay we’ve ever had simultaneously affecting both sites.

The good news is that Hawaiian Telecom made good progress repairing the fiber on the slopes of Haleakala today, and should get us back online by Friday at the latest–assuming they can secure a helicopter to assist. Apparently, helicopters don’t grow on trees, even in Hawaii.

The probably-not-good news is that, here in CA, the weather is still bad and who knows when it will clear up. Forecast says rain for another week, but maybe some of that could be during the day and the clouds could push out at night? 🙂

Just because you’re in Hawaii, doesn’t mean it’s warm! We got our first frost of the season, after being socked in and unable to observe last night.

We can see effects of cooler temperatures in California, too. One of the camera’s in RFO2 is especially prone to being sticky when cold, but fortunately we now have software to handle it automatically.

Ideally, and with most of the other cameras, it takes only 3 attempts to open the shutter all the way 3 times in a row, but last night it took 9.

For the first time in years, I took a trip to Maui where the primary reason was actually to have fun. But of course I wasn’t going to miss the opportunity to visit LaserSETI! And my father-in-law was kind enough to come along for the ride and even give me a hand–literally and figuratively.

It’s always a pleasure, after long drive up the mountain as the last step of the journey, to climb the stairs of the Airglow Building and see the instruments, peacefully sitting amongst the incredible scenery.

The main bit of work we did, besides inspection and maintenance check, was to install a 19V step-up transformer, because we’ve found that the USB bus on this newer generation of NUC computers can be a little finnicky when supplied with 12V like everything else in the instrument. Since the science cameras and disk are connected via USB, any issues in that department are big issues.

Checking in on the air filters, it was clearly time to replace them. Apparently summer is a dusty time up there on the summit. These MERV 13 filters aren’t cheap, but they do a their job very well and have demonstrated a very reasonable operational life expectancy.

In every LaserSETI instrument, there’s a large-capacity USB hard drive that we use to capture all the data coming from the two science cameras. Even with the biggest drives being made today, we can only store a month or two worth of data, but it’s an important staging area. The one in IFA2, however, had been dropping offline whenever we tried to read or write too much data to it.

So we shipped out a new drive to Doug, who’s local on Maui, and yesterday he drove the 3-hour round trip to the summit and replaced the drive. The weather had been uncooperative and we prefer not to operate in the rain, but he saw an opening and made a dash for it.

I don’t want to say “it’s fixed” too early or jinx it, but last night we made it through a full night of observing, streaming data at over 100 Mbps, while also backhauling the data to our datacenter in California… stressing the hard drive to the max, and everything went smoothly! Fingers crossed it stays happy for a good long while.

Turns out I did save a “before” movie of the RFO1 sunshade that wasn’t opening fully! It had been working great for a long time, but the real world is messy like that apparently. Clearly it was the humidity, cold temperatures, phase of the moon, and local unicorn population that was causing the friction.

Speaking of locals, Dan, did a phenomenal job this weekend and got the main computer within RFO2 back online! It’s been a long string of wacky issues, from the exact DC voltage it was receiving, to the firewall, to the unusual Vadj potentiometer on our DC transformer. The science cameras are still offline, but that’s probably a minor issue and we’ll hope to be observing again this week.

In related news, the hard drive on IFA2 has been flaky, so we’re also hoping to get that replaced on a similar timeline.