Difference between revisions of "HG-WELS Spring work"

From CoolWiki
Jump to navigationJump to search
m
 
(2 intermediate revisions by one other user not shown)
Line 4: Line 4:
 
=Task One: Wiki & Excel & Angles & ds9 training=
 
=Task One: Wiki & Excel & Angles & ds9 training=
  
Go and watch the videos on the wiki, linked from the top of this page: [[Guide to NITARP participants for use of the wiki]]. Quick victory!
+
Go and watch the videos on the wiki, linked from the top of this page: [[Guide to NITARP participants for use of the wiki]]. Quick victory! Try editing something (your user page, or, say, the [[HG-WELS Spring Calendar]] page to update your planned outages.
  
 
Are you comfortable moving between plain text and Excel files? This always seems to cause a lot of problems, with every single NITARP team I have ever worked with, but I don't know how to get around it. IPAC table format is used a lot with IPAC-related archives and tools, but it has a file extension not commonly accepted by either Windows or Mac machines, so you have to know how to help your computer deal with these files.  If you are not super, super comfortable with this, '''you need to be''', especially in the context of your (most likely Windows) laptop and how it handles these files. Watch [https://www.youtube.com/watch?v=nCJ3ctOGvNk this video] on moving between tbl files and xls. This was originally created for a 2011 team, so it contains some extraneous information, but the fundamental principles still apply (and you can tell I'm trying to be very, very patient and very, very complete in my explanations... because I have explained this so. many. times. And so many teams have gotten this explanation at one time and completely forgotten everything about it 2 months later.).  
 
Are you comfortable moving between plain text and Excel files? This always seems to cause a lot of problems, with every single NITARP team I have ever worked with, but I don't know how to get around it. IPAC table format is used a lot with IPAC-related archives and tools, but it has a file extension not commonly accepted by either Windows or Mac machines, so you have to know how to help your computer deal with these files.  If you are not super, super comfortable with this, '''you need to be''', especially in the context of your (most likely Windows) laptop and how it handles these files. Watch [https://www.youtube.com/watch?v=nCJ3ctOGvNk this video] on moving between tbl files and xls. This was originally created for a 2011 team, so it contains some extraneous information, but the fundamental principles still apply (and you can tell I'm trying to be very, very patient and very, very complete in my explanations... because I have explained this so. many. times. And so many teams have gotten this explanation at one time and completely forgotten everything about it 2 months later.).  
Line 19: Line 19:
 
=Task Two: Assemble Target Lists=
 
=Task Two: Assemble Target Lists=
  
We need to take the source lists we have from de la Reza, Carlberg's paper, and Carlberg's email and merge it into one master catalog, one line per source, keeping track of where the sources appear. This catalog should include RA and Dec. This catalog can be plain text or Excel, either one, but since you and your students are most likely going to be using Excel, might as well start now to assemble the catalog into Excel.
+
We need to take the source lists we have from de la Reza, Carlberg's paper, and Carlberg's email and merge it into one master catalog, one line per source, keeping track of where the sources appear and keeping track of any and all ancillary data that these folks have already assembled for us. This catalog should include RA and Dec at the very minimum, but should also have columns of, say, VJHK, from the contributing catalogs. This catalog can be plain text or Excel, either one, but since you and your students are most likely going to be using Excel, might as well start now to assemble the catalog into Excel.
  
We need to have an [http://irsa.ipac.caltech.edu/applications/TblValidator/ IPAC Table] version of this catalog so that we can use it to search in IRSA services, so once we have the Excel spreadsheet assembled, we will need to *export* a clean text file from Excel that can be interpreted as a tbl file.  
+
We need to have an [http://irsa.ipac.caltech.edu/applications/TblValidator/ IPAC Table] version of this catalog so that we can use it to search in IRSA services, so once we have the Excel spreadsheet assembled, we will need to *export* a clean text file from Excel that can be interpreted as a tbl file. (This table can consist of just name, ra, dec, because that's the minimum needed for searching, but our full catalog should have many more columns in whatever format you prefer.)
  
 
de la Reza et al. 1997 has this table: http://iopscience.iop.org/1538-4357/482/1/L77/fulltext/5120.text.html#tbls
 
de la Reza et al. 1997 has this table: http://iopscience.iop.org/1538-4357/482/1/L77/fulltext/5120.text.html#tbls
Line 43: Line 43:
  
 
Make columns of decimal ra, dec if needed. Convert the ra properly to degrees.
 
Make columns of decimal ra, dec if needed. Convert the ra properly to degrees.
 +
 +
RA (degrees) = (hh*15)+(mm/4)+(ss/240); Dec = Degrees + arcmin/60 + arcsec/3600, watch negatives!
 +
Here's an RA/Dec converter to help verify values, or use with students:
 +
https://www.swift.psu.edu/secure/toop/convert.htm
  
 
Make a new file (or tab) that contains just name, and decimal ra, dec. Export that as plain text, not tab-separated or comma-separated. This may take some fussing. Make the right header for an IPAC table. Send your table through the [http://irsa.ipac.caltech.edu/applications/TblValidator/ IPAC table validator] to make sure it's got the right format, and save the reformatted one to disk.
 
Make a new file (or tab) that contains just name, and decimal ra, dec. Export that as plain text, not tab-separated or comma-separated. This may take some fussing. Make the right header for an IPAC table. Send your table through the [http://irsa.ipac.caltech.edu/applications/TblValidator/ IPAC table validator] to make sure it's got the right format, and save the reformatted one to disk.

Latest revision as of 05:00, 18 March 2014

Also see HG-WELS Spring Calendar

Task One: Wiki & Excel & Angles & ds9 training

Go and watch the videos on the wiki, linked from the top of this page: Guide to NITARP participants for use of the wiki. Quick victory! Try editing something (your user page, or, say, the HG-WELS Spring Calendar page to update your planned outages.

Are you comfortable moving between plain text and Excel files? This always seems to cause a lot of problems, with every single NITARP team I have ever worked with, but I don't know how to get around it. IPAC table format is used a lot with IPAC-related archives and tools, but it has a file extension not commonly accepted by either Windows or Mac machines, so you have to know how to help your computer deal with these files. If you are not super, super comfortable with this, you need to be, especially in the context of your (most likely Windows) laptop and how it handles these files. Watch this video on moving between tbl files and xls. This was originally created for a 2011 team, so it contains some extraneous information, but the fundamental principles still apply (and you can tell I'm trying to be very, very patient and very, very complete in my explanations... because I have explained this so. many. times. And so many teams have gotten this explanation at one time and completely forgotten everything about it 2 months later.).

The video does not cover *exporting* the file as a plain text file, but you can do that too. And we will.

In coming weeks, we will need to be familiar with angles on the sky, so please review angular measures on the sky using that link or your own textbook of choice. Think about how you would convert RA in hours to RA in degrees.

Finally, we will also need ds9. Here are ds9 Tutorials from the official NITARP tutorial (Jan 2013):

Task Two: Assemble Target Lists

We need to take the source lists we have from de la Reza, Carlberg's paper, and Carlberg's email and merge it into one master catalog, one line per source, keeping track of where the sources appear and keeping track of any and all ancillary data that these folks have already assembled for us. This catalog should include RA and Dec at the very minimum, but should also have columns of, say, VJHK, from the contributing catalogs. This catalog can be plain text or Excel, either one, but since you and your students are most likely going to be using Excel, might as well start now to assemble the catalog into Excel.

We need to have an IPAC Table version of this catalog so that we can use it to search in IRSA services, so once we have the Excel spreadsheet assembled, we will need to *export* a clean text file from Excel that can be interpreted as a tbl file. (This table can consist of just name, ra, dec, because that's the minimum needed for searching, but our full catalog should have many more columns in whatever format you prefer.)

de la Reza et al. 1997 has this table: http://iopscience.iop.org/1538-4357/482/1/L77/fulltext/5120.text.html#tbls which gives just source names, not coordinates. I took these source names and ran them automagically through a service here at IPAC that resolves the names using SIMBAD. SIMBAD tabulates data, but is notoriously unreliable at doing so. Don't believe any Vmags, types, or classifications of objects you find as part of their 'basic data.' Their coordinates are pretty good, and SIMBAD provides synonyms to objects and links back to articles (and the data tables therein), though, which are, in general, far more useful. They also provide links to lists of objects found in any given paper. In SIMBAD, you can also search by position, so you can find out, e.g., what other named objects are near to a region you care about, or synonyms for your object. There may be other useful papers calling objects near your targeted region by those other names, rather than the name you're using; if you search in ADS just by one name, you may miss papers calling the object by a different name, and this is not an issue when you search in SIMBAD. SIMBAD also does pretty well at, given a name, getting good coordinates for it, providing it can find the name.

Here: File:Delarezawcoords.txt is my automatically generated version, with IPAC- and SIMBAD-resolved coordinates. Did I grab all the objects (e.g., compare total number of sources in the de la Reza table and mine)? Do all of them have coordinates (hint: there is a gotcha in there)? Spot check a few (e.g., pick 5 randomly from the list, not just the top 5) by searching on their names manually in SIMBAD. Did I get the coordinates right? Can you figure out how to fill in the coordinates for the missing object(s)? http://simbad.harvard.edu/simbad/sim-ref?querymethod=bib&simbo=on&submit=submit+bibcode&bibcode=1997ApJ...482L..77D is the SIMBAD version of the data table from de la Reza et al. 1997. (Did you see this at the top of the page? Could be useful!)

Save a copy of this table in Excel, and then copy it into a new file called something to the effect of "master list." In your Excel version of this master list file, add a column that indicates that objects here came from de la Reza 1998 Table 1. We will be adding in rows from other tables, and it will be important to remember that these objects came from this table. As an aside, note that these are the kinds of instructions I will give you, now and through this project: "add a column", as opposed to "select all of column G, go up to the edit menu, choose add column to create a new column G" (and, for that matter, "save the file" as opposed to "save it with this filename in this location"). The idea here is that you've got to understand what it is you're looking at, what columns are what, where you're keeping files, how you will name the files, and keep notes so that you understand what is really going on. There are no cookbooks here. This has frustrated people in the past, so I am calling it to your attention now - this is not me "being vague" or "not providing guidance" .. this is trusting you as a collaborator to understand what we are doing and keep track of things in a way that makes sense to your workflow.


Carlberg mailed us a copy of her target list from her paper. I believe it already had coordinates, as well as other information such as V magnitudes. We will want to keep all this ancillary data along for the ride (we will need it later). Read this file into an Excel file, save it once you get it into Excel, and then and merge it into your now master copy of the source list. You can do it however you want; I would do it like:

  • Read it in, save it as Excel on its own (so that if you screw something up in the next few steps, you still have a version with all the columns aligned).
  • Select all the cells in the Carlberg table, with headers, copy.
  • paste into the master copy, with the headers, below the existing data.
  • move around the columns JUST from the new data such that the cells that are RA/dec from Carlberg actually line up with the existing ra/dec columns. You will probably have to shift things right enough cells so that you have space to do this.
  • move the headers around so that the new columns from carlberg have headers that line up at the top with the other headers, and then you can delete the headers that you pasted in to the middle of the file.
  • add values to the column indicating the origin of the object (in this case, 'Carlberg12', or whatever you want to denote this origin), and 'fill down' to indicate the origin of these objects
  • sort by ra. look by eye, or create a new column that is ra minus the next row's ra, and conditionally color the ones for which that value is very small so that you can find them. Are there duplicates? Duplicates by different names but not positions? Merge the information, row by row, preserving information and origin, and delete the duplicate rows. If the positions are different, retreat to SIMBAD to get coordinates, and make a note of this. There are IRSA tools we can use to resolve some of these issues.

Repeat for the second catalog Carlberg sent. There may very well be duplicates in there with respect to de la Reza.

Make columns of decimal ra, dec if needed. Convert the ra properly to degrees.

RA (degrees) = (hh*15)+(mm/4)+(ss/240); Dec = Degrees + arcmin/60 + arcsec/3600, watch negatives! Here's an RA/Dec converter to help verify values, or use with students: https://www.swift.psu.edu/secure/toop/convert.htm

Make a new file (or tab) that contains just name, and decimal ra, dec. Export that as plain text, not tab-separated or comma-separated. This may take some fussing. Make the right header for an IPAC table. Send your table through the IPAC table validator to make sure it's got the right format, and save the reformatted one to disk.

Task Three: Resolution issues

We need to start to develop some instincts about spatial resolution, specifically IRAS and WISE. HG-WELS Resolution Worksheet