노정훈

The power of PyData stack is built upon the ability of Numpy and Pandas to push basic operations into lower-level compiled code via an intuitive higher-level syntax.While these abstractions are effcient and effective for many common use cases, they often rely on the creation of temporary intermediate objects, which can cause undue(=excessive) overhead in computational time and memory use. To add..

Pandas was originally developed in the context of financial modeling, so it contains an extensive set of tools for working dates, times, and time-indexed data.Timestamps : Particular moments in timeTime interval and periods : A length of time between a particular beginning and end pointTime deltas or durations : An exact length of timeDates and Times in PythonNative Python Dates and Times: datet..

Introducing Pandas String OperationsVectorization of operations simplifies the syntax of operating on arrays of data.For array of strings, Numpy does not provide simple access, and thus you're stuck using a more verbose(=long-winded) loop syntax.# In[1]data=['peter','Paul','MARY','gUIDO'][s.capitalize() for s in data]# Out[1]['Peter', 'Paul', 'Mary', 'Guido']This is perhaps sufficient to work wi..

The pivot table takes simple column-wise data as input, and groups the entries into a two-dimensional table that provides a multidimensional summarization of the data.Pivot tables as essentially a multidimensional version of groupby aggregation.That is, you split-apply-combine, but both the split and the combine happen accross not a one-dimensional index, but across a two-dimensional grid.Motiva..

# In[1]class display(object): """Display HTML representation of multiple objects""" template=""" {0}{1} """ def __init__(self,*args): self.args=args def _repr_html_(self): return '\n'.join(self.template.format(a,eval(a)._repr_html_()) for a in self.args) def __repr__(self): return '\n\n'.join(a+'\n'+repr(eval(a)) ..

# In[1]class display(object): """Display HTML representation of multiple objects""" template=""" {0}{1} """ def __init__(self,*args): self.args=args def _repr_html_(self): return '\n'.join(self.template.format(a,eval(a)._repr_html_()) for a in self.args) def __repr__(self): return '\n\n'.join(a+'\n'+repr(eval(a)) ..

# In[1]def make_df(cols,ind): data={c:[str(c)+str(i) for i in ind] for c in cols} return pd.DataFrame(data,ind)# In[2]class display(object): """Display HTML representation of multiple objects""" template=""" {0}{1} """ def __init__(self,*args): self.args=args def _repr_html_(self): return '\n'.join(self.template.format(a,eval(a)._repr_html_()) ..

A Multiply Indexed SeriesBad Way# In[1]index=[('California',2010),('California',2020),('New York',2010),('New York',2020),('Texas',2010),('Texas',2020)]populations=[37253956,39538223,19378102,20201249,25145561,29145505]pop=pd.Series(populations,index=index)pop# Out[1](California, 2010) 37253956(California, 2020) 39538223(New York, 2010) 19378102(New York, 2020) 20201249(Texas, 20..

Trade-offs in Missing Data ConventionsA number of approaches have been developed to track the presence of missing data in a table or DataFrame.They revolve around one of two strategies: using a mask that globally indicates missing values, or choosing a sentinel value that indicates a missing entry.In the masking approach, the mask might be an entirely separate Boolean array, or it might involve ..

Pandas inherits much of the functionality from Numpy, including ufuncs.Pandas will automatically align indices when passing the objects to the ufunc.This means that keeping the context of data and combining data from different sources-both potentially error-prone tasks with raw Numpy arrays- becomes essentially foolproof with Pandas.Ufuncs: Index PreservationBecause Pandas is designed to work wi..