[Commotion-admin] [mlab-observatory] Various updates (#33)

mtlynch notifications at github.com
Mon Jan 5 16:26:13 EST 2015


I wrote a bunch of code over the holiday slowdown thinking that I could use git magic later to group the commits into neat, cohesive pull requests, but it turns out it's not so easy to do that so my pull requests got a little convoluted and I apologize. This pull request does the following:

* Fixes sample_checking.py to use the `SingleTelescopeResultReader` class to parse Telescope input files (previously it used the `parse_filename_for_metadata` and `parse_data_file` but these are now module private - insofar as naming conventionsl let us pretend things are private).

* Changes `TelescopeResultReader` so that it's an iterator over all the parsed rows (like the Python file `open` API is with lines of text) instead of the caller having to call `read_rows`. Now functions that are expecting a list of raw Telescope data rows can receive a `TelescopeResultReader` and not know the difference.
  * As a result of the above, we update `sample_checking.py` to just pass along a `TelescopeResultReader` object when it calls `add_to_counts()` instead of having to explicitly call `read_rows`.

You're welcome to review the pull request commit-by-commit if it's easier (though I will generally try to prevent making pull requests where you have to do that because it's usually a pain)
You can merge this Pull Request by running:

  git pull https://github.com/mtlynch/mlab-observatory reader-iterator

Or you can view, comment on it, or merge it online at:

  https://github.com/opentechinstitute/mlab-observatory/pull/33

-- Commit Summary --

  * Fixing sample_checking so that it calls new APIs for parsing Telescope data
  * Changing the semantics of TelescopeResultReader so that now _read_rows is private and callers get the rows through the __iter__ function. Also got rid of the get_metadata function in MergedTelescopeResultReader because it was confusing and unncessesary.
  * Updating DataFileBlacklister to use new TelescopeResultReader iterator.

-- File Changes --

    M convert_from_telescope/result_merge.py (37)
    M convert_from_telescope/result_merge_test.py (90)
    M convert_from_telescope/sample_checking.py (17)
    M convert_from_telescope/telescope_data_parser.py (34)
    M convert_from_telescope/telescope_data_parser_test.py (33)

-- Patch Links --

https://github.com/opentechinstitute/mlab-observatory/pull/33.patch
https://github.com/opentechinstitute/mlab-observatory/pull/33.diff

---
Reply to this email directly or view it on GitHub:
https://github.com/opentechinstitute/mlab-observatory/pull/33
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.chambana.net/pipermail/commotion-admin/attachments/20150105/26a60cb4/attachment.html>


More information about the Commotion-admin mailing list