The ECHO project is part of an ongoing effort to use embedded RFID tags in penguins in order to gather information about when individual penguins have returned to their colony. ECHO is a wheeled robot which can read penguin RFID tags by staying near the penguin colony. The robot has an RFID antenna mounted on the front, and has a sensor suite which will allow it to make decisions about which penguins to approach. ECHO represents a potentially huge improvement because it can stay near the colony without human interference and perform in a greater variety of weather conditions.
My contribution to the ECHO project has been to write the software package which ECHO uses while near the emperor penguin colony to approach the individuals and scan their tags. This work included the fusion of ECHOs various sensors, including a LiDAR, a 360 camera, the RFID antenna as well as the robot's odometry from motor feedback and an IMU. I developed a custom neural network to find penguins in the robot's live camera feed, and wrote a filtering algorithm for the LiDAR to pull out individual penguins from the raw point cloud. Fusing these inputs allowed the robot to determine where the nearby penguins were. I implemented a tracking system to determine which penguins had already been scanned for an RFID tag, wrote the speed controller for approaching and disengaging the penguins, and created a collision avoidance routine.
After developing the navigation package in simulation I traveled to Neumayer III station near the penguin colony at Atka Bay in order to test the package and gather data on its performance during the 2023-2024 field season. I spent three months in Antarctica and was able to conduct 25 days worth of outdoor testing on ECHO while supporting the other ongoing projects taking place at Neumayer.
Performing some testing of the CNN used for penguin detection on a stormy day at Neumayer.
The neural net shown during field tests, differentiating between human researchers and penguins laying on their bellies.
Example of the data output from the LiDar. Each color indicates an individual penguin, and the ground plane is shown in grey.
The above shows bounding boxes shown for each of the penguins found in the LiDar feed.
Setting and tracking a target object.
Robot perspective of identifying a target object for scanning.
The Whale Spotter is a device used to autonomously detect whales in the path of vessels in order to avoid collisions. Vessel strikes are one of the leading causes of whale injury and death, especially among the critically endangered North Atlantic Right Whale.
For this project, I took a proof-of-concept prototype whale spotter and redesigned it in order to scale the device for mass production. This included a redesign of all the subsystems, such as the camera stabilization, enclosure, electronics, networking, and computer system. I also performed half a dozen device deployments, led trainings for vessel crews, and ran in-field testing on multi-week cruises in various parts of the world.
Please note - due to intellectual property concerns I have not included detailed imagery of the electronics subsystems, but will be happy to discuss the work.
Full whale spotter system I installed and deployed on a shipping vessel transiting the pacific ocean. More information about this collaboration can be found here.
The whale spotter works by using thermal cameras to detect whale blows in front of a vessel.
This interface is shown on a tablet on the bridge of a vessel. This shows the time of a whale detection, as well as the approximate location.
This assembly drawing indicates the custom designed gimbal used to stabilize the two thermal cameras.
The camera system stabilizes in both roll and pitch to account for the motion of the vessel. An inertial measurement unit feeds a PID controller, which in turn drives the motors. Each systems PID parameters are tuned before shipping.
Drawing demonstrating one of the electronics subsystems.
This drawing demonstrates the enclosure designed to eliminate water ingress, while reducing condensation.
Silicone gaskets used to prevent water ingress and are resistant to UV degradation.
Shown here with an older housing design.
Through my work at WHOI, I have performed several remote ecosystem monitoring installation projects. I have shared a few examples here, including a whale spotter installation in Northern Canada, a whale spotter installation on an active ferry system, and some of my Antarctic field work.
Currently, I am working on an Autonomous Surface Vehicle (ASV) which will be used for offshore ecosystem monitoring. The vehicle is a modified WAM-V, which uses natural gas as a supplementary fuel source to improve the range of the vehicle.
For this project, I have sourced and integrated the sensor suite which will be used to identify and count birds and marine mammals offshore. I have also written the autonomous pipeline which the vehicle uses offshore to make decision about where to perform surveys and when to return to shore. Lastly, I have integrated the sensor suite, the autonomy and the onboard controller to prepare for testing in late 2025.
The ASV will work in tandem with underwater gliders and buoys in order to collect data about the whole ecosystem nearby wind farm installations and other offshore infrastructure.
This chart gives a sense of the coverage provided by the sensor suite onboard the ASV and accessory vessels.
I modeled this track using MOOS-IVP.
Calculating the sensor specs to ensure optimal coverage while collecting seal data.
LiDar data taken near WHOI shows a ship departing from dock and gives us a sense of the expected resolution from the LiDar.
This is the custom PCB used to control the generator and motors onboard the ASV and communicate with the HLC.
The purpose of this project was to design a set of custom rubber treads for the ECHO chassis. This improved maneuverability on ice and prevented the robot from getting stuck in the snow while driving.
For this project, I designed the hardware to make the treads compatible with our off-the-self Clearpath Husky body, found an international supplier to manufacture the treads, rebuilt the rover with the new hardware, and prepared it for another season of testing in Antarctica.
When I traveled to Neumayer Station with ECHO to perform autonomy testing, I observed that the wheeled design led to difficulties in mobility and getting stuck in the snow. After the field season, I redesigned the robot to move to a treaded design.
I explored other treaded robot chassis and inquired with Clearpath about a modified Husky chassis with treads, but ultimately found it would be faster and more economical to design custom treads for our own Husky unit.
First tests after the new treads arrived and were assembled, evaluating the turn radius and slipping.
A critical aspect of this project was facilitating communication with a custom tread supplier over seas. To facilitate this process, I created dozes of drawings to explain critical dimensions and overcome the language barrier.
Detailed drawing used for the tread sprockets.
Solidworks model of the idler plate assembly, highlighting the reversible feature of the design. This allows the tread tension to be configured manually, with either a crowned or flat contact plane.
In preparation for the field season, I also redesigned the top plate and electronics access panel to make sensor mounting and serving easier.
ECHO running at Neumayer with my custom treads, shown with the updated top plate and sensor mounts.
As a part of my work at WHOI, I try to perform outreach as often as possible to share our work and highlight the importance of climate and ocean research. These are a few of the most fun outreach events I have led.
My work on the whale spotter project is featured in the Apple TV natural history project ‘The Wild Ones’ in their episode on North Atlantic Right Whales. During filming, I was able to collect hundred of confirmed NARW sightings on our AI system to use for training data.
Giving an on screen overview of the Whale Spotter system to camera trap expert Declan Burley.
The rest of the science team on board during the filming of The Wild Ones.
I accepted a role as a visiting scientists on a cruise vessel traveling from the southern point of Argentina to the Antarctic Peninsula via the Falkland Islands and South Georgia Island.
Installing a whale spotter unit onboard the cruise vessel so that I could take whale detection data for the duration of the cruise.
As a visiting scientist, I performed studies on the wildlife as well as answering questions about the environment and animals from the passengers.
As part of the science outreach, I gave this talk on the whale spotter project.
Giving an ECHO demonstration and answering penguin questions at a local childrens museum.
Preparing ECHO to be shown in a GEO documentary showcasing various extreme environments and the scientists working there.
Interviewing with Dirk Steffens, representing German documentary organization GEO.
