pytesmo.validation_framework.upscaling module

class pytesmo.validation_framework.upscaling.MixinReadTs

Bases: object

Mixin class to provide the reading function in DataAverager and DataManager

read_ds(name, *args)

Function to read and prepare a datasets.

Calls read_ts of the dataset.

Takes either 1 (gpi) or 2 (lon, lat) arguments.

Parameters:

• name (string) – Name of the other dataset.

• args (either gpi or (lon, lat)) –

  • gpi (int): Grid point index

  • lon (float): Longitude of point

  • lat(float): Latitude of point

Returns:

data_df – Data DataFrame.

Return type:

pandas.DataFrame or None

class pytesmo.validation_framework.upscaling.Upscaling(ref_class, others_class, upscaling_lut, manager_parms)

Bases: MixinReadTs

This class provides methods to combine the measurements of validation datasets (others) that fall under the same gridpoint of the dataset being validated (reference).

The goal is to include here all identified upscaling methods to provide an estimate at the reference footprint scale.

Implemented methods:

• time-stability filtering

• simple averaging

Parameters:

• ref_class (<reader object> of the reference) – Class containing the method read_ts for reading the data of the reference

• others_class (dict) – Dict of shape {‘other_name’: <reader object>} for the other dataset

• upscaling_lut (dict) – Dict of shape {‘other_name’:{ref gpi: [other gpis]}}

• manager_parms (dict) – Dict of DataManager attributes

get_upscaled_ts(gpi, other_name, upscaling_method='average', temporal_stability=False, **kwargs) → None | DataFrame

Find the upscale estimate timeseries with given method, for a certain reference gpi

Parameters:

• gpi (int) – gpi value of the reference point

• other_name (str) – name of the non-reference dataset to be upscaled

• upscaling_method (str) –

  method to use for upscaling:

  • ’average’ takes the simple mean of all timeseries

• temporal_stability (bool, default is False) – if True, the values are filtered using the time stability concept

• kwargs (keyword arguments) – arguments for the temporal window or time stability thresholds

Returns:

upscaled – upscaled time series; if there are no points under the specific gpi, None is returned

Return type:

pd.DataFrame or None

static temporal_match(to_match, hours=6, drop_missing=False, **kwargs) → DataFrame

Temporal match to the longest timeseries

Parameters:

• to_match (list) – list of dataframes to match

• hours (int) – window to perform the temporal matching

• drop_missing (bool, optional. Default is False.) – If true, only time steps when all points have measurements are kept

Returns:

matched – dataframe with temporally matched timeseries

Return type:

pd.DataFrame

static tstability_filter(df, r_min=0.6, see_max=0.05, min_n=4, **kwargs) → DataFrame

Uses time stability concepts to filter point-measurements (pms). Determines whether the upscaled measurement based on a simple average of all the pms is in sufficient agreement with each pm, and if not eliminates pm from the pool.

Thresholds are based on: Wagner W, Pathe C, Doubkova M, Sabel D, Bartsch A, Hasenauer S, Blöschl G, Scipal K, Martínez-Fernández J, Löw A. Temporal Stability of Soil Moisture and Radar Backscatter Observed by the Advanced Synthetic Aperture Radar (ASAR). Sensors. 2008; 8(2):1174-1197. https://doi.org/10.3390/s80201174

Parameters:

• df (pd.DataFrame) – temporally matched DataFrame

• r_min (float) – lower threshold for correlation (Pearson) between upscaled and pm

• see_max (float) – upper threshold for standard error of estimate

• min_n (int) – minimum number of pms to perform the filtering

Returns:

filtered – filtered input

Return type:

pd.DataFrame

upscale(df, method='average', **kwargs) → Series

Handle the column names and return the upscaled Dataframe with the specified method.

Parameters:

• df (pd.DataFrame) – Dataframe of values to upscale using method

• method (str) – averaging method

• kwargs (keyword arguments) – Arguments for some upscaling functions

Returns:

upscaled – dataframe with “upscaled” column

Return type:

pandas.DataFrame