Cloud Cover

HyPlan Clouds

Overview: Optical remote sensing of the Earth’s surface often requires clear skies. Deploying airborne remote sensing instruments can be costly, with daily costs for aircraft, labor, and per diem travel expenses for aircraft and instrument teams. This script addresses the question: “Statistically, how many days is it likely to take to acquire clear-sky observations for a given set of flight boxes?”.

The script operates under several simplifying assumptions: - Each flight box is “flyable” in a single day given clear skies from a single base of operations. - Instantaneous MODIS Terra/Aqua overpasses are representative of clear-sky conditions throughout the flight day. - Other environmental state parameters (e.g., tides, wind speeds) do not influence go/no-go decisions for flights.

Key Features: 1. Cloud Data Processing:

  • Reads geospatial polygon data (GeoJSON) representing flight areas for airborne optical remote sensing.

  • Fetches MODIS cloud fraction data from Google Earth Engine for specified years and day ranges.

  • Aggregates daily cloud fraction data for each polygon.

  1. Flight Simulation:

    • Simulates daily flight schedules to visit polygons based on a maximum cloud fraction threshold.

    • Enforces constraints such as maximum consecutive flight days and optional weekend exclusions.

  2. Visualization:

    • Generates heatmaps of cloud conditions, visit days, and rest days for each simulated year.

    • Produces cumulative distribution function (CDF) plots to estimate the likelihood of completing visits.

TODO: Currently, it is not possible to span years (a campaign can not include dates in both December and January).

get_binary_cloud(image)[source]

Generates a binary cloud mask for a given MODIS image.

The MOD09GA/MYD09GA state_1km band encodes cloud state in bits 0-1:

00 = clear, 01 = cloudy, 10 = mixed, 11 = not set.

Any non-zero value (bits 0-1 != 00) is treated as cloudy.

Parameters:

image (ee.Image) – An Earth Engine image with a “state_1km” quality assessment band.

Returns:

Binary cloud mask (1 for cloudy/mixed, 0 for clear) with an added “date_char” property.

Return type:

ee.Image

calculate_cloud_fraction(image, polygon_geometry)[source]

Calculates the cloud fraction over a given polygon for a MODIS image.

Parameters:

  • image (ee.Image) – An Earth Engine MODIS image.

  • polygon_geometry (ee.Geometry) – A polygon geometry representing the region of interest.

Returns:

A feature containing the date and calculated cloud fraction for the polygon.

Return type:

ee.Feature

create_date_ranges(day_start, day_stop, year_start, year_stop)[source]

Creates date ranges for filtering Earth Engine image collections.

Parameters:

  • day_start (int) – Start day of the year (1-365).

  • day_stop (int) – End day of the year (1-365).

  • year_start (int) – Start year for the ranges.

  • year_stop (int) – End year for the ranges.

Returns:

A list of date range tuples (start_date, end_date) in YYYY-DDD format.

Return type:

list of tuples

create_cloud_data_array_with_limit(polygon_file, year_start, year_stop, day_start, day_stop, limit=5000)[source]

Processes MODIS cloud data for polygons and calculates daily cloud fractions.

The polygon file must contain a ‘Name’ column identifying each polygon.

Parameters:

  • polygon_file (str) – Path to a GeoJSON or shapefile containing polygons with a ‘Name’ column.

  • year_start (int) – Start year for data processing.

  • year_stop (int) – End year for data processing.

  • day_start (int) – Start day of the year for data processing.

  • day_stop (int) – End day of the year for data processing.

  • limit (int, optional) – Maximum number of images to process per date range. Default is 5000.

Returns:

A DataFrame with columns ‘polygon_id’, ‘year’, ‘day_of_year’, and ‘cloud_fraction’.

Return type:

pd.DataFrame

simulate_visits(df, day_start, day_stop, year_start, year_stop, cloud_fraction_threshold=0.1, rest_day_threshold=6, exclude_weekends=False, debug=False)[source]

Simulate visits to polygons based on cloud fraction thresholds, ensuring no more than one visit per day. Adds rest days after a set number of consecutive visits and resets counters on weekends or when no polygons meet the threshold.

On each visitable day, the alphabetically first unvisited polygon that meets the cloud threshold is chosen. Rest days count toward total_days but no polygon is visited.

Parameters:

  • df (pd.DataFrame) – Cloud fraction data with columns: ‘polygon_id’, ‘year’, ‘day_of_year’, ‘cloud_fraction’.

  • day_start (int) – Start day of the year for simulation.

  • day_stop (int) – End day of the year for simulation.

  • year_start (int) – Start year for simulation.

  • year_stop (int) – End year for simulation.

  • cloud_fraction_threshold (float) – Maximum allowable cloud fraction for a visit.

  • rest_day_threshold (int) – Maximum number of consecutive visits before a rest day is required.

  • exclude_weekends (bool) – If True, skip weekends and reset the counter for rest days.

  • debug (bool) – If True, enable detailed logging for debugging.

Returns:

  • DataFrame summarizing total days simulated per year.

  • Dictionary of visit days for each polygon, organized by year.

  • Dictionary of rest days for each year.

Return type:

Tuple[pd.DataFrame, Dict[int, Dict[str, list]], Dict[int, list]]

plot_yearly_cloud_fraction_heatmaps_with_visits(cloud_data_df, visit_tracker, rest_days, cloud_fraction_threshold=0.1, exclude_weekends=False, day_start=1, day_stop=365)[source]

Generates heatmaps of cloud fraction for each year, including visit markers and rest day highlights.

Parameters:

  • cloud_data_df (pd.DataFrame) – DataFrame with columns ‘polygon_id’, ‘year’, ‘day_of_year’, and ‘cloud_fraction’.

  • visit_tracker (dict) – A dictionary of visit days for each polygon, organized by year.

  • rest_days (dict) – A dictionary of rest days for each year.

  • cloud_fraction_threshold (float) – Threshold to classify cloud fraction as clear (white) or cloudy (black).

  • exclude_weekends (bool) – If True, weekends are highlighted and skipped in the heatmap.

  • day_start (int) – Start day of the year to include in the heatmap.

  • day_stop (int) – End day of the year to include in the heatmap.

Returns:

Displays heatmaps for each year with clear/cloudy days, visit days, and rest day markers.

Return type:

None