pastebin - collaborative debugging

pastebin is a collaborative debugging tool allowing you to share and modify code snippets while chatting on IRC, IM or a message board.

This site is developed to XHTML and CSS2 W3C standards. If you see this paragraph, your browser does not support those standards and you need to upgrade. Visit WaSP for a variety of options.

financialpython private pastebin - collaborative debugging tool What's a private pastebin?


Posted by DK on Thu 29 Oct 04:38
report abuse | download | new post

  1. #!/usr/bin/python
  2.  
  3. '''The following script cleans up data from the yfd tab-delimited file in a
  4. cursory way and creates a numpy array of duration, start, and end time.
  5. '''
  6.  
  7. import numpy as np
  8. import csv
  9. import pytz
  10. from pytz import timezone
  11. from datetime import datetime, timedelta
  12.  
  13. file = open('ethandataOct28.txt')
  14. file.readline()
  15.  
  16. #create numpy array of data
  17. types = {'names':('name','value','unit','time'),
  18.          'formats':('S10', 'S5', 'S5', 'S20')}
  19.  
  20. ethan = np.loadtxt(file, dtype=types, delimiter='\t')
  21.  
  22. sleep_mask = np.logical_or(ethan['name']=='gnight', ethan['name']=='gmorning')
  23. sleep_index = np.where(sleep_mask)[0]
  24. sleep_raw = ethan[sleep_mask]
  25.  
  26. #find potential errors in data (i.e., there are 2 gnight's in a row)
  27. errors_index = np.where(sleep_raw['name'][:-1] == sleep_raw['name'][1:])
  28. errors = sleep_raw[errors_index]
  29.  
  30. if errors.size != 0:
  31.     print "There are duplicate gnight/gmorning entries. Check the data!"
  32.  
  33. #create array of cleaned data
  34. sleep_clean = np.column_stack((sleep_raw['name'], sleep_raw['time']))
  35.  
  36. time_stringlist = sleep_clean[:,1].tolist()
  37.  
  38. time_objectlist=[]
  39. for item in time_stringlist:
  40.     time_objectlist.append(datetime.strptime(item, "%Y-%m-%d %H:%M:%S"))
  41.    
  42. time_objectarray = np.array(time_objectlist)
  43.  
  44. sleep_events = np.column_stack((sleep_clean[:,0], time_objectarray))
  45.  
  46. #a little logic to make sure I calculate time asleep and not time awake
  47. if sleep_events[0][0] == 'gmorning':
  48.     gmorning_objects = sleep_events[::2,1]
  49.     gnight_objects = sleep_events[1::2,1]
  50. else:
  51.     gmorning_objects = sleep_events[1::2,1]
  52.     gnight_objects = sleep_events[2::2,1]
  53.  
  54. sleep_durations = gmorning_objects - gnight_objects
  55. duration_table = np.column_stack((sleep_durations,
  56.                                   gnight_objects, gmorning_objects))

Submit a correction or amendment below (click here to make a fresh posting)
After submitting an amendment, you'll be able to view the differences between the old and new posts easily.

Syntax highlighting:

To highlight particular lines, prefix each line with @@


Remember me so that I can delete my post