View source for Working with the IC 417 data


''This page is an updated version of the [[Working with the C-CWEL data]] page (and to some extent the [[Working with the HG-WELS data]] page), which was an update of the [[Working with the C-WAYS data]] page, which was an update of the  [[Working with the BRCs]] page, which was an update of the [[Working with CG4+SA101]] page, which was an update of the [[Working with L1688]] page. '''This''' page was developed and updated specifically for the 2015 IC 417 team visit.  

'''Please note''': '''NONE''' of these pages are meant to be used without applying your brain! They are NOT cookbooks!''  This is presented as a linear progression because of the nature of this page, but we have already done some things "out of order", and moreover, chances are excellent that you will go back and redo different pieces of this at different stages of your work.

FOR REFERENCE: [[IC 417 Bigger Picture and Goals]]

FOR REFERENCE: [[IC 417 Box Disk Contents]]. Includes instructions on how to force your computer to read any files with an extension you don't recognize (.tbl, .reg).

=Useful Positions=
<font color="red">(just for reference)</font>

We are studying a square that is ~1 deg on a side, centered on 5:28:00 34:30:00. 

=Obtaining the imaging data=

<font color="red">DONE</font>

We found imaging data for this region from WISE, 2MASS, Spitzer/IRAC (GLIMPSE), UKIDSS, IPHAS, and Jose et al. 2008, but we can only get our hands on imaging data from DSS, WISE, 2MASS, GLIMPSE, and IPHAS. 

We have already used FinderChart and other IRSA tools. We have already used Skyview at Goddard. We have already used ds9.

'''Big picture goal''': Getting images and viewing them in ds9 or IRSA Viewer.

'''Here and now''': Either re-pull FITS images for yourself for our region in DSS, 2MASS, and WISE, or get them from on the Box drive.
Bonus: Spitzer/IRAC (GLIMPSE), and IPHAS. (I don't have images for UKIDSS.)

'''Relevant links''': 
*[[How do I download data from WISE?]] 
*[http://irsa.ipac.caltech.edu/applications/wise/  Access the WISE archive directly]. 
*http://irsa.ipac.caltech.edu/Missions/wise.html WISE
*http://irsa.ipac.caltech.edu/Missions/2mass.html 2MASS
*[http://skyview.gsfc.nasa.gov/cgi-bin/skvadvanced.pl Goddard's Skyview]
*[http://nitarp.ipac.caltech.edu/resource/8 ds9 NITARP Tutorial] (listed on that page, along with installation tips)
*[http://hea-www.harvard.edu/RD/ds9/site/Download.html ds9 Download site]
*http://irsa.ipac.caltech.edu/data/SPITZER/GLIMPSE/ GLIMPSE
*http://www.iphas.org/ IPHAS

=Investigating the big mosaics =

'''It is "real astronomy" to spend a lot of time staring at the mosaics and understanding what you are looking at. Don't dismiss this step as not "real astronomy" just because you are not making quantitative measurements.  This is time well-spent, and you should plan on investing some time doing this section.'''  Some aspects of this were already discussed in the context of the Resolution worksheet.

'''Big picture goal''': Recognize what is an instrumental artifact and what is real. Understand what is seen at each WISE and Spitzer band and all the other archival bands.

'''More specific shorter term goals''': Understand what is part of the sky and what is an artifact (e.g., not part of the sky).  Recognize how the images differ among the various bands, and why. 

'''Relevant links''':  
*[[What is a mosaic and why should I care?]] 
*Possibly [[Making Mosaics Using MONTAGE]]. 
*[http://skyview.gsfc.nasa.gov/cgi-bin/skvadvanced.pl Goddard's Skyview]
*[[Resolution]] and associated [[IC 417 Resolution Worksheet]]. 
*Why does it matter to know what is an artifact and what is not? [http://www.universetoday.com/86497/proof-bio-station-alpha-is-just-an-image-artifact/ So you don't get fooled by stuff like this.] 
*[[How can I make a color composite image using Spitzer and/or other data?]]
*[http://nitarp.ipac.caltech.edu/resource/8 ds9 NITARP Tutorial] (listed on that page, along with installation tips)
*[http://hea-www.harvard.edu/RD/ds9/site/Download.html ds9 Download site]

'''Here and now''': Load the images into a viewer of your choice, ds9 or [http://irsa.ipac.caltech.edu/fftools/app.html IRSA Viewer].

'''Questions for you''': 
#'''MOST IMPORTANT of these questions: Compare the mosaics across the bands. What changes? What stays the same? Why? (This is a DEEP question! See also next questions.)'''
#How does the number of '''stars''' differ across the bands? Which band has the most stars? The fewest? (Bonus question: why?) The most '''nebulosity'''? The least? (Bonus question: why?) Are there more stars in the regions of nebulosity, or less? Why? 
#What is saturated? Are the same objects saturated in all bands? What are some other instrumental effects you can see? 
#Notice the pixel scale. Which survey has the lowest resolution (biggest pixels)? (Bonus: is that the same as the native pixels for the survey? You will need to Google, or go back to your [[IC 417 Resolution Worksheet]] notes.
#Make a three-color image. Do the stars match up? Does the nebulosity? 
#BONUS: How big are any of the features in the image (nebulosity, galaxy, space between objects)? (What do I mean by big?) in pixels, arcseconds, parsecs, and/or light years? (Hint: you need to know how far away the thing is. If it helps, there are 3.26 light years in a parsec.)

=Obtaining the catalog data and bandmerging across catalogs=

<font color="red">DONE</font>

We found data for this region from WISE, 2MASS, Spitzer/IRAC (GLIMPSE), UKIDSS, IPHAS, and Jose et al. 2008. 

We have already used FinderChart and other IRSA tools to retrieve 2MASS and WISE (and Spitzer) catalogs. 

'''Big picture goal''': Get the data you need for this project. Get you comfortable enough to search for your own favorite target in WISE, understand what to do with the search results, and download data. 

'''More specific shorter term goals''': Obtain the relevant catalogs.

'''Relevant links''': 
*[[How do I download data from WISE?]] 
*[http://irsa.ipac.caltech.edu/applications/wise/  Access the WISE archive directly]. 
*http://irsa.ipac.caltech.edu/Missions/wise.html WISE
*http://irsa.ipac.caltech.edu/Missions/2mass.html 2MASS
*http://irsa.ipac.caltech.edu/data/SPITZER/GLIMPSE/ GLIMPSE
*http://www.ukidss.org/ UKIDSS
*http://www.iphas.org/ IPHAS

What I did: 
#Download catalogs from these sources over our region.
#Using a computer, load in catalog1. Then, for each of the sources in catalog2, metaphorically sit on each source in catalog2 and look for a match in catalog1. If I find a match, associate those sources. If I do not find a match, sometimes I added the entire source to the catalog, and sometimes I just dropped it. (There are a LOT of sources here, many we do not care about, so having each and every source in here is less important than it might be.)
#Now have catalog1+catalog2. Do same for each source in catalog3, such that I merge in catalog3 with catalog1+2. Repeat for catalog4, etc.
#Given ensemble catalog, look for matches with Xavier's list of interesting sources. Tag them in the database.
#Repeat for Jose et al. 2008 Halpha stars, OB stars, and IPHAS' list of Halpha stars.
The process of merging the bands across catalogs is called "bandmerging."

After this process, I have about 29,000 objects in a master catalog, about 200 of which we care about. Remember that we are interested in the ENTIRE set of {things Xavier tagged as possibly young from the AllWISE catalog} PLUS {things Xavier tagged as possibly young from his own PhotVis reprocessing of the WISE data} PLUS {things Jose et al. tagged as Halpha stars} PLUS {things Meyer & Macak tagged as OB stars} PLUS {things IPHAS tagged as bright in Halpha}. 

'''Questions for you''': 

#How do you download catalogs for any of these surveys?

=Previously identified sources=

<font color="red">DONE.</font>

You've already really done this prior to the visit, but this is where it kind of falls in the logical progression here. (I see the progression as : pick target, get data, get numbers for the sources in the images first from the literature and where others have done it for us, then do more ourselves, then make plots, etc.) 

'''Big picture goal''': Understand what has already been studied and what hasn't in the region we care about.

'''Relevant links''': 
*[[How can I find out what scientists already know about a particular astronomy topic or object?]] 
*[[I'm ready to go on to the "Advanced" Literature Searching section]] 

'''Process''':
#Search ADS, SIMBAD.
#Identify literature of relevance.
#Read literature.
#Extract from it the data we care about.
#For data tables of sources obtained via non-electronic detectors (even some electronic detectors), assess how good the positions are. Can we blindly match these sources to the ensemble catalog (which has positions better than an arcsec)?
#If not, use FinderChart to investigate each source by hand to 'correct' its position to be one that can be merged blindly with the rest of the catalog.
#Then, merge in the literature observations (in this case, optical multiwavelength catalog from Jose et al for everything they detected), and conclusions about objects (in this case, list of Halpha stars from two places, list of OB stars in essence from 2 places). 
#Tag the interesting objects as interesting in the database so we can find them again.

'''IMPORTANT''': when we are done with all of this, at the end of the process, for the sources we have settled on as 'interesting', we should get into SIMBAD and search for each of our sources, just to get another handle on whether or not anyone has done anything on them before -- because there just might be something else already identified in the literature.

=Data Tables=

You may have already have developed some of these skills via the [http://coolwiki.ipac.caltech.edu/index.php/IC_417_Summer_visit_logistics#Excel_to_know |shortlist of stuff I sent in April]. If not, this is the time to learn!

'''Big picture goal''': Learn how to manipulate data tables. Look at the data tables to identify what you need. Start to cope with source list proliferation. 

'''More specific shorter term goals''': Understand how to import plain text tables into Excel (or another spreadsheet of your choice). Really look at the files as retrieved from the WISE archive and look at files as sent by Xavier and determine the relevant information out of these catalogs. Keep track of which files are which.
  
'''Relevant links''': 
*[http://www.youtube.com/watch?v=nCJ3ctOGvNk YouTube video] on what tbl files are, how to access them, and specifically how to import tbl files into xls. (10min)  

Note that many data tables come with many, many, many lines (like more than 100) at the top explaining what the contents of the file are. These are useful for keeping with the file (like a FITS header is useful to keep with the image), but when reading it into Excel, you may wish to delete all but a note to yourself about what the file is, and the headers of the data columns themselves.

'''Tasks and Questions for you''': 
#Practice for getting catalogs on your own: Get a WISE catalog for our region, either from a download or the DVD. Find a way to read the file, perhaps by loading it into Excel. Cope with the long header - scan it, keep it elsewhere, but you may not want to read it in to your Excel copy. What information do we need? How do you figure out which ones have bad photometry? You will need this in Excel for the next steps, so you will need to do this.
#I have already merged the catalogs from WISE, 2MASS, Spitzer/IRAC (GLIMPSE), UKIDSS, IPHAS, and Jose et al. 2008. The entire catalog has ~29,000 sources. This pretty much brings Excel to its knees. There is some utility in having the entire data set in order to make plots of 'everything' in order to see how the YSOs stand out as different. I dumped the entire catalog into a text file for you if you want it. 
#I have a short list of ~200 sources that are our focus for our work. This one you should load into Excel for sure. This catalog has all the photometric data I have for each source, but also an indication of why the source is on the list of interesting sources. Values for the "whyhere" column are combinations of 2-letter codes:
***xp = xavier found it from PhotVis processing (x=Xavier, p=PhotVis)
***xw = xavier found it from the allWISE processing (x=xavier, w=wise)
***ha = Jose et al found it (and you corrected the positions for it) because it is an Halpha star (ha=Halpha)
***ih = IPHAS tagged it as bright in Halpha (=possibly young)
***ob = Meyer & Macak found it (and you corrected the positions for it) because it is an OB star (ob=OB)
You will find some like 'xpxwha', which means Xavier found it in both of his processings (xp,xw), *and* it is an Halpha star (ha). There are many that have just one code.
*'''CAUTION 1:''' There are files from me with everything in the region (long) and files with just the things in which we are interested (short) -- meaning the literature-identified plus the new Xavier-identified. 
*'''CAUTION 2, AND THIS ONE'S A BIGGIE''': '''These catalog files generally have a mixture of detections and limits, measurements and errors, flux densities and magnitudes.''' You will need to be careful in importing this into Excel. The data are all Vega mags; some have errors and some have limits.

=Making color-color and color-magnitude plots=

OK, fair warning, some math involved, and the start of ''programming'' spreadsheets!

'''Big picture goal''': Understand what plots to make. Understand the basic idea of using them to pick out certain objects.

'''More specific shorter term goals''': Make some color-color and color-mag plots using the complete WISE catalog imported into Excel in the last step. At the very least, make a plot of the entire distribution and highlight the objects of interest in the plot. 

'''Relevant links''': 
*[[Color-Magnitude and Color-Color plots]] 
*[[Gutermuth color selection]] - mostly currently Gutermuth color selection; includes analogy with M&Ms. Koenig color selection is similar in concept but uses different bands.
*[[Finding cluster members]] 
*[[Color-color plot ideas]] 
*Also see slides from my set of talks on Monday.

'''Tasks and Questions for you''': 
#To do these next steps, you need the full WISE catalog from above, and the catalog of interesting objects from me. If you did all the steps in the prior task, you have this now; I also have a copy of this for you on the DVD.
#Start from the complete WISE catalog. Pick at least one color-color or color-magnitude plot to make (Suggestions: W1 vs W1-W4, W1-W2 vs W3-W4).  Figure out a way to ignore the "no data" flags (exactly what they are depends on which file you are starting from - they could be 'null' or they could be '-9'). Does the photometry seem ok? Identify the regions where you expect to find the plain stars and the IR excess objects.  
#Which objects are on our shortlist of interesting objects? Overplot them '''on the same plot''' as above with a different color and/or shape symbol. 
#EXTRA BONUS, especially when circling back to this step - are there any sources you can identify in this diagram that might be worth a second look in SEDs or images?

Notes: For a W1-W2 vs W3-W4 plot. W1-W2 is (should be) centered on zero, because most of the objects seen in W1 and W2 are plain stars, so the color is zero. W3-W4 is notably NOT zero, because the only objects seen at W4 are the ones notably bright at W4, so they all are brighter than plain stars at W4. This is going to be a different morphology than a, IRAC color-color diagram (I1-I2 vs I3-I4 plot), where a much larger number of sources are seen, and a large fraction of those are plain stars. This gave me heart failure during the 2012 summer visit until I realized this. YSO candidates are bright and red, generally. There are other CMDs you can try. See any of the literature we read while writing our proposal for ideas.  After we include some optical data, there will be even more CMDs we can try. We will come back to this step.

=Making SEDs =

WARNING: lots of math and programming spreadsheets here too... '''you WILL do this more than once to get the units right!'''

'''Big picture goal''': Understand how to convert magnitudes back and forth to flux densities. Understand what an SED is and why it matters.

'''More specific shorter term goals''': Program a spreadsheet to convert between mags and flux densities.
Make at least one SED yourself.  

Make sure you understand how to get the fluxes from the magnitudes. This is not easy to do right the first time, so you '''will''' get the wrong answer the first few times you try. 

'''Relevant links''': 
*[[Units]] 
*[[SED plots]] 
*[[Central wavelengths and zero points]]
*[[Studying Young Stars]]  

We will ultimately need to make SEDs for everything, but for purposes of this example, let's work with these three objects off the first few on our shortlist: 052532.00+345835.7, 052532.08+345815.2, 052532.62+344000.0.  Start with just one. You will ultimately plot log (lambda*F(lambda)) vs log (lambda) -- see the [[Units]] page. It will take time to get the units right, but once you do it right the first time, all the rest come along for free (if you're working in a spreadsheet). Spend some time looking at the SEDs. Look at their similarities and differences. Identify the bad ones, and discuss with the others why/whether to drop them off the list of YSO candidates.  See also stuff above about data at other wavelengths, and include literature/archival data from other sources where appropriate and possible. Make sure to keep careful track of those things that are limits rather than detections.

Another try at explaining:
*''What do you have?'' UBVRIriHalpha, JHK, WISE, Spitzer data in Vega mags. 
*''What do you need to get?'' everything into Jy, which are units of Fnu -- look up how to convert between mags and flux density ([[Units]] page and [[Central wavelengths and zero points]]). Then convert your Fnu in Jy into Fnu in cgs units, ergs/s/cm2/Hz, so multiply by 10^-23. Then convert your Fnu into Flambda in cgs units, so multiply by c/lambda^2, with c=2.99d10 cm/s and lambda in cm (not microns!).  Then get lambda*Flambda by multiplying by lambda in cm.  Plot log (lambda*Flambda) vs. log (lambda).
*Once you make your first SED correctly, the rest are easy. But that first one is hard.
*Then you need to look through each of the SEDs and decide which look like you expect, which need photometry to be checked, and which seem unlikely to be legitimate YSOs. This is a judgement call, and your judgement will improve with time as you gain some experience.

You can do all of this in one massive spreadsheet such that you do the calculations for all ~200 SEDs at once. This is the power of Excel. Or, you can make one at a time. (You will probably need to plot one at a time anyway, because stupid Excel.)

'''Questions for you''':
#What do the IR excesses look like in your plots?  Do they look like you expected? Like objects in Monday's ppt or elsewhere?
#'''If you made all the SEDs at once:''' For comparison, find these objects and look at their SEDs: 052743.28+343156.5 052839.40+344008.5. Do you expect them to have a large excess based on [3.4]-[22]? What do they look like? Why are we considering these objects?

=Assessing SEDs =

Now that you've made some SEDs, we need to next look at them with a critical eye. 

'''Big picture goal''': Understand what to expect in a YSO SED and how to discard objects for having questionable SEDs (or put them on the list for checking source matching, photometry, etc).

'''More specific goals''': Examine the SEDs for all of our candidate objects. Use them to further evaluate the quality of the YSO candidates from the YSO candidate list.

'''Relevant links''': 
*[[Studying Young Stars]]  
*the detailed object-by-object discussion in the appendix of the [http://lanl.arxiv.org/abs/1105.1180 cg4 paper]. 

Get the file with all the SEDs in it from me. We will do the first 9 as a group, but then you should work as a team and go through each of the objects and make notes about what you see. Does it look like a YSO SED? Does it look like the data weren't tied to the correct source or there were spatial resolution problems? Discuss with your partner about whether or not you believe each source, and why. Don't forget to compare your notes on SEDs with notes on images (e.g., if it's tagged "iffy" in images and "iffy" in SEDs, chances are excellent that it is not a good candidate).

'''Questions for you''':
#Which objects look like clear YSO SEDs? Which objects do not? Can you learn anything about this from the image inspection? Keep good notes on this!

By this point, you will have a set of objects in which we are interested, and you should have (will have) notes on each of the objects, obtained from the image check and the SED check. Collate all of these such that you can tag objects as "unlikely to be real YSOs", or "literature YSOs (that may or may not have a disk)", or "still surviving as new YSO candidates".

=Revisiting CMDs and CCDs=

'''Goal:''' We have more data, so we should use it to refine our list of true YSO candidates vs. junk.

By this point you should have a list of things that have survived tests. We have optical data for a lot of these objects. You can make optical CMDs and see if the "literature YSOs (that may or may not have a disk)" and "still surviving as new YSO candidates" fall in the 'right place' in these diagrams. For context, take all the optical data we have for all the objects in the region and plot r vs r-i from the IPHAS data. Overplot the survivors on this diagram. Are they where they should be (above the ZAMS)? Do this again, but for r-Halpha vs. r-i.   Are the survivors where they should be (substantially above the unreddened main sequence locus)? Tag any of the survivors as less likely if they don't meet these criteria.

=Analyzing SEDs=

'''This is advanced, and we may not get here.'''  

Add a new column in Excel to calculate the slope between 2 and 8 microns in the log (lambda*F(lambda)) vs log (lambda) parameter space. This task only makes sense for those objects with both K band and IRAC-4 detections.  (For very advanced folks: ''fit'' the slope to all available points between K and IRAC-4 or MIPS-24.  How does this change the classifications, if at all?)
*if the slope > 0.3 then the class = I
*if the slope < 0.3 and the slope > -0.3 then the class = 'flat'
*if the slope < -0.3 and the slope > -1.6 then class = II
*if the slope < -1.6 then class = III
These classifications come from Wilking et al. (2001, ApJ, 551, 357); yes, they are the real definitions  ([[Studying Young Stars|read more about the classes here]])! 
#How many class I, flat, II and III objects do we have?
#Where are the objects with infrared excesses located on the images? Are all the Class Is in similar sorts of locations, but different from the Class IIIs?

For very advanced folks: [http://cfa-www.harvard.edu/youngstars/dalessio/ suite of online models from D'Alessio et al.] and [http://caravan.astro.wisc.edu/protostars/ suite of online models from Robitaille et al.].  Compare these to the SEDs we have observed.

=Putting this in context a little: Methodology=


'''Big picture goal''': Understand how what we did is different than what Jose et al. (2008) did with the data to find young stars.

'''More specific shorter term goals''': Knowing what you do now, go back and reread Jose et al. (2008). Make a detailed comparison of our method for finding young stars and that from the Jose et al. paper. 

'''Relevant links''': [[How can I find out what scientists already know about a particular astronomy topic or object?]] and [[I'm ready to go on to the "Advanced" Literature Searching section]].

'''Questions for you''':
#What are the steps (cookbook-style) that Jose et al. used to find YSOs?
#What were our steps? 
#How are they different?  
#Did we recover all of the young stars identified in the literature? Some will not have IR excesses, so those will not be recovered by an IR-based color selection.

=Putting this in context a little: Science=

'''Goal:''' put our work in context with other literature and other star-forming regions from the literature.

How many YSOs with IR excess did we see? How many literature YSOs did not have IR excesses? Do we have any evidence that the YSOs from the literature are not actually YSOs? How many new YSOs did we see? What is the fraction of Class I/flat/II/III? Where do they fall on the sky, e.g., can we see things that are likely to be members of the clusters that Camargo et al. laid out?

=Writing it up!=

'''Goal:''' We need to write an AAS abstract and then the poster, and if we're lucky, a paper!

We need to include:
#How the data were taken.
#How the data were reduced.
#What the IR properties are of the previously identified YSOs here, in context with other observations from the literature.
#What the IR properties are of the new sources we have found, including objects you think are new YSOs (or objects you think are not), and why you think that.
#How this region compares to other regions observed with Spitzer or WISE.

Take inspiration for other things to include from other papers on star-forming regions in the literature.