Skip to content

Index Monotonic Sort Bug in class DateChunker  #974

@qcyfred

Description

@qcyfred

Index Monotonic Sort Bug in class DateChunker (in file date_chunker.py)

If the df's index is not monotonic increasing, arctic will sort the df by index. BUT the variable dates is still not in order.

I suggest arctic to put the code dates = df.index.get_level_values('date') after the if sentence.

def to_chunks(self, df, chunk_size='D', func=None, **kwargs):
    """
    chunks the dataframe/series by dates

    Parameters
    ----------
    df: pandas dataframe or series
    chunk_size: str
        any valid Pandas frequency string
    func: function
        func will be applied to each `chunk` generated by the chunker.
        This function CANNOT modify the date column of the dataframe!

    Returns
    -------
    generator that produces tuples: (start date, end date,
              chunk_size, dataframe/series)
    """
    if 'date' in df.index.names:
        dates = df.index.get_level_values('date')
        if not df.index.is_monotonic_increasing:
            df = df.sort_index()
        # TODO dates won't be sorted, which will cause data store error.
        
      # dates = df.index.get_level_values('date')

Anyway, arctic is an excellent project !

这是我第一次在github上留言。蹩脚的英文。

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions