Using autopiloted drones to measure floating kelp

Background

Drones are perceived as a way to efficiently and accurately measure change in the volume of floating kelp plants (kelp) in Puget Sound, the Salish Sea and beyond. An image from a drone-borne camera may be used to count the kelp bulbs that are floating at the water's surface, assuming that each bulb represents a living kelp plant. A seasonal or annual sequence of images of an area of sea water may be used to calculate change, if any, in the volume of kelp plants in that area. This approach can be labeled as "low-altitude plant sampling" (LAPS).

 

However there are many variables to consider when designing and implementing a long-term, scientifically valid LAPS approach. This article summarizes observations from testing parts of a LAPS approach over a two-year period, and is offered to stimulate discussion, funding and action to protect, restore and potentially extend floating kelp patches in the Salish Sea and along other coastlines. Initial effort was in coordination with the Northwest Straits Commission. Several alternative approaches are mentioned toward the end of this ariticle.

 

From the current viewpoint of the US Federal Aviation Administration, a "drone" has been defined as "an unmanned aircraft system (UAS) ... that is controlled from an operator on the ground". However many other terms and definitions are being used in the rapidly-evolving, global unmanned aircraft industry. Tests for this article were done with a system that included:

  • an FAA-licensed operator;

  • a programmable, autopiloted, multirotor drone;

  • two, detachable digital cameras;

  • equipment and software for programming or controlling the drone;

  • a laptop computer with relevant software.
     

The drone's cameras recorded low-altitude images from time to time during two growing seasons of a patch of floating kelp (Nereocystis luetkeana) near the shoreline of Ebey's Landing National Historic Preserve on Whidbey Island in Washington State.

While important, the following topics are not fully explored in this summary:

  • The value of floating kelp canopies to humans, fish and other life on Earth

  • Current laws and regulations that are intended to protect floating kelp plants

  • Factors or causes affecting, or projected to affect the overall volume of floating kelp plants

    • Land use; boating; storms; grazers; displacements; imbalances; ocean acidification, ​... 

  • Prior, planned or funded research about ecosystems that include a canopy of floating kelp

  • Public or private organizations responsible for protecting or restoring floating kelp abundance

  • Georeferencing of physical kelp patches, beds or plants to a map, image, model or dataset

  • Preferences for terms and definitions to be used in research about floating kelp (patch, bed) 

  • Use of images from camera and other sensors on satellites, airplanes, helicopters

  • Use of images from camera and other sensors on land masses near to areas with floating kelp

  • Collection of in situ data (physical samples obtained through using boats, submarines, divers)

  • Applications of a LAPS approach to monitoring of eelgrass, noxious weeds or other plants

  • Reference websites, databases, software applications or other information sources, for example the Marine Vegetation Atlas maintained by the Washington State Department of Natural Resources

  • Consumer, commercial and military industries associated with UAS technologies and uses

  • Unmanned Traffic Management (UTM) across US State, International and other boundaries (for example: Washington State - Oregon State; United States - Canada).

  • Comparison of a LAPS approach to other approaches.

Problem

  • Are canopies of floating kelp decreasing across Puget Sound and the Salish Sea?

One approach to developing a valid answer is to periodically record images from a low altitude above the kelp plants that are appearing in the surface of the sea water, then count the number of visible kelp bulbs. Over time, the data should indicate any decreases (or increases) in the number of healthy kelp plants in a monitored area. Based on long-term data from a collection of monitored areas, extrapolations may be made to describe larger environmental trends across Puget Sound, the Salish Sea and beyond.

 

However, many variables must be recognized and accommodated for this approach to yield valid scientific results, for example:

  • What is counted as a single kelp bulb? How much of the bulb must be above the water surface? What if the bulb is partially covered by a kelp frond?

  • Is the counted object the bulb of a living kelp plant, and not some other object?

  • Is the counted bulb entirely within the boundaries of the monitored area of sea water?

  • Over time, were the images of the monitored area recorded from the same airspace positions, at the same altitude, with the same camera, lens and filter, and with the same angle relative to nadir (vertical to the water surface)?

  • Over time, was the same geospatial framework used as a reference for identifying the boundary edges and corners of the images?

  • Over time, were the images recorded during similar environmental conditions? Similar water current speed and direction? Angle of sunlight on the water? Water ripples, swells or waves?

  • Over time, was the recorded data about environmental conditions based on local measurements at the monitored area, or based on forecasts for a larger geographical area?

  • Over time, were the counts of bulbs in each image done by using the same configuration of software, with the same parameters and value settings?

  • Over time, could the biases of individual researchers or volunteers have had an affect on the consistency of the counting process for each image?

Challenges to establishing a consistent kelp monitoring approach

Long-term monitoring to measure change in a scope of a natural environment can be difficult and costly. A combination of in situ, forecasting and remote-sensing techniques may be used. Remote-sensing techniques can quickly record valuable data without intervention in the area being studied, however there are limits to the kinds of data that can be obtained. The image below is an example of how sun glint, algae, debris and masses of tangled kelp plants can reduce visibility of objects in sea water, which affects the accuracy of counts of kelp bulbs.

Is it a living kelp bulb, or is it a log, resting sea bird, kelp frond, kelp stipe, or debris? A specially-tuned infrared camera can help reveal the living kelp bulbs from among other other visible objects.

Is it a living Nereocystis luetkeana kelp bulb, or the bulb of some other species of floating kelp? Advanced multispectral cameras and digital image processing software may be able to use a calibrated "signature" or "profile" of reflected light from the intended plant to automatically identify the species, and to separate plants from other objects. The second image below is an example (note how the floating log that is visible in the first image disappears from view in the second image).

To be counted, was the kelp bulb entirely within the boundaries of the monitored area? The direction and speed of water currents within the monitored area can significantly affect which bulbs are within the area at the time an image was recorded. Bulb counts may also be affected by:

  • water ripples, swells or waves

  • sun glints resulting from the angle of sunlight on the water and kelp plants

  • perceptions of individual researchers or volunteers.

What is meant by "the monitored area"? Here are three possibilities:

  1. An area of sea water as seen by a person at a site that includes kelp plants.

  2. A snapshot image of an area of sea water as recorded by a camera sensor. 

  3. An area of sea water as established by some other method.

This writing focuses on (2.).

The approach described in this writing depends upon using a small, multirotor drone that can hover to maintain its horizontal and vertical position and directional orientation in airspace, while an electronic gimbal maintains the camera in horizontal position at a fixed angle from vertical. Repeatability of camera positioning depends on the drone and gimbal's ability to perform in a variety of wind conditions.

 

The author's drones have been upgraded to lock to L1 signals from GPS, GLONASS and Galileo satellites, with the internal GNSS receiver typically reporting a horizontal dilution of precision (DOP) of 0.6 meters (however vertical DOP is less). An upgrade to post-processed kinematic (PPK) or real-time kinematic (RTK) performance levels is available to achieve a horizontal DOP of less than 10 centimeters.

Over time, there must be a stable specification and consistent realization of images of the monitored area. Two options include:

  1. The coordinates of the boundaries (corners and edges) of the image, relative to a geospatial framework, effectively represent the boundaries of the monitored area;

  2. The coordinates of the boundaries of the monitored area are identified some distance within the boundaries of the image.

For simplicity, this writing focuses on option (1.). 

 

For consistent realization of images of the monitored area, several factors should be considered:

  • Is the drone capable of holding a consistent position in airspace while monitoring an area during typical weather conditions for the area?  

  • Were the images recorded at the same altitude, with the same camera, lens and filter, and with the same angle relative to nadir (vertical to the water surface)?

  • Were the images aligned to a geospatial framework?

  • Were the images developed as mosaics from other images through a software-controlled process? If so, did the "stitching" process introduce anomalies within the image, or produce different results when performed at different times?

  • Was human intervention required to edit the image with the intent of improving the accuracy of the bulb count?

For example, the image below is a mosaic from many images that were recorded from over a beach and an area of sea water that included floating kelp. The image stitching process that was used produced many anomalies/errors, the visual impact of which was minimized through manual editing. However the process that was used cannot be expected to result in undistorted and accurately georeferenced images for long-term monitoring of changes in floating kelp. Even with geotagging of the source images, there were insufficient visual tie-points for accurate automatic processing. That said, advancements in image capture and processing methods may improve the quality of stitched images of floating kelp to be sufficient for scientific purposes. Until then, the author recommends avoiding stitching of images.

Which altitude is best for recording images of floating kelp with an autopiloted drone? The answer depends on finding a balance among these objectives:

  • Sufficient image resolution for accurate counting of kelp bulbs within a digital image.

  • Absence of image blurring due to motion of the drone, the camera in its electronic gimbal, or the kelp plants.

  • Airspace conditions and a programmed drone speed that allows a flight duration sufficient for launch, arrival above the monitored area, recording of the image, and returning to a landing position.
     

For example, the following images were recorded with a stock MAPIR Survey 2 NDVI Red+NIR camera at 20 feet above the surface of the water. 

When is the best time to record images of a monitored area? For the most reliable bulb counts, it appears that images should be obtained:

  • early in the growing season, prior to significant clumping or intertwining of kelp plants, and prior to warmer days when algae blooms are likely, and

  • at identical tide levels, and

  • at similar water current speed and direction, and

  • during similar water ripples, swells or waves, and

  • at a time of day when the angle of sunlight is similar to previous monitoring events.


The above requirements are based in part on the following observations:

  • After the bulbs of new plants emerge at the water surface at lower tide levels, the plants continue to grow, resulting in longer stipes which can swing in many directions with changes in water current speed and direction. The direction and speed of water currents within the monitored area can significantly affect which bulbs are within the area at the time an image is recorded.

  • The volume of visible floating kelp plants increases during the annual growing season, then decreases over winter. However plants from previous seasons may be mixed with the new plants.

  • Algae may attach to, and obscure bulbs and other portions of kelp plants.

  • At different times after the start of a growing season (and specific to a selected area) clumps or masses of kelp plants will appear on the surface at lower tide levels, inhibiting accurate counts of kelp bulbs.

  • If water ripples, waves or swells are large enough, the visibility of countable kelp bulbs can be reduced.

Alternative approaches

 

Assuming that the primary goal of a monitoring project is to accurately measure change in the volume of floating kelp canopies,​​ the following approaches could be explored:

  • Monitor from a fixed-wing or hybrid VTOL drone. Larger kelp canopy areas can be scanned at low altitudes by small drone aircraft flying at higher speeds, however the quality of the images will depend on the capability of the camera, lens and memory card. Fixed-wing drones typically require a larger launch and landing area. Hybrid VTOL drones offer the possibility of hovering at launch, transitioning to fixed-wing flight, then transitioning to hover for landing.

  • Monitor from an airplane or helicopter. Piloted aircraft may be used to record images of floating kelp areas. As with images from autopiloted aircraft, the images must be accurately aligned to a geospatial framework. Images from piloted aircraft are typically recorded at higher altitudes, which may limit the achieved image resolution and the ability to accurately count kelp bulbs. 

  • Monitor by using satellite images. This may be a viable alternative if an annual or seasonal time-series of images can be obtained consistently during clear weather conditions, and with sufficient resolution to detect changes in the size of a bounded area of floating kelp. 

  • Monitor from shore. Establish a survey-grade monument and a high-quality, attachable tripod with a  calibrated infrared camera. Rotate and tilt the camera to aim at the approximate center of an area of kelp, with a field-of-view that is taller than the far edge of the typical maximum growth area of the canopy. Count the pixels in the image that are within acceptable values. Compare pixel counts over a season, and annually. This approach may not be feasible in locations where the monument cannot be reliably placed, or where the monument/tripod/camera is not high enough above the water's surface to indicate expansions or contractions of the visible area of the canopy.

  • Monitor from a boat, using cameras or human observers. While there are some advantages to boat-based approaches, it can be difficult to maintain a boat's position and directional orientation with high accuracy, particularly in less-than-calm weather conditions and/or while among floating kelp plants. Inaccurate or unstable positioning of the boat reduces the accuracy of bulb counts when using snapshot images. Also, later in the season there are masses of tangled kelp plants that can inhibit accurate bulb counting because of the low angle of perception of a camera or human observer.

Conclusions

The selection of sites to be monitored long-term will be affected by the level of consistent funding for equipment, training, equipment maintenance and repair, field operations and office operations.

 

The author has found that sites with floating kelp can vary significantly across Puget Sound. Many, but not all sites are reachable by car and on foot. Some areas have a beach for easier drone launches and landings, while others are rocky and steep which raises the risk of an accident, especially during a drone landing. Also, some areas of floating kelp may not be reachable with any multirotor drone, only by using a hybrid VTOL (vertical takeoff or landing) or fixed-wing drone.

Below is a brief outline of steps that may be followed to begin long-term monitoring of one area of floating kelp by using a UAS that includes an autopiloted, multirotor drone with an advanced GNSS receiver.

  1. Identify the general area of sea water to monitor, in part by using images from satellites, piloted airplanes, or piloted helicopters.

  2. Identify the approximate time period each year within which all of the following conditions are likely to be met, in or near the bounded area to be monitored:

    • A quantity of bulbs of new plants have emerged at the water surface at a lower tide level.​

    • An acceptable drone launch and landing area is available within approximately 1500' of the area to be monitored. (As of July 2017, drone operators are required by the FAA to have line-of-sight visibility of the drone. Exceptions may be granted by the FAA.).

  3. For the time period selected in (2.) above, identify the times of day when the angle of sunlight minimizes sun glint or glare from the water or kelp plants.

  4. Identify the altitude of the camera on the drone, that will result in images with resolution that enables accurate bulb counting. The acceptable altitude can be identified by analyzing images that were recorded while the drone hovered at various altitudes over the emerging kelp.

  5. Identify a position over the kelp that is expected to reveal changes in the volume of local kelp plants.

  6. Program the drone's autopilot to travel at an altitude to a waypoint in airspace over an area of kelp, while positioning the camera gimbal in nadir (or near-nadir) position. 

  7. Record one or more images over one minute (or another period of time), to enable estimation of the effects of any shifting of the camera's actual field of view as affected by wind conditions or the drone's position as continually recalculated by the internal GNSS receiver (number and constellation of satellite signals locked; variability from solar disturbances).

  8. Use software to evaluate the quality of the images, and to auto-count the acceptable bulbs appearing in the images. Select the representative image for documentation of the monitoring event. Verify with researchers, volunteers or other observers.

  9. The annual or seasonal images may be integrated into a layer in a geographic information system (GIS) specifically for documenting changes in the volume of floating kelp.