Working with the C-WAYS data - Revision history

Rebull at 19:56, 24 July 2012

2012-07-24T19:56:04Z

Rebull at 20:06, 17 July 2012

2012-07-17T20:06:40Z

Rebull at 04:45, 12 July 2012

2012-07-12T04:45:29Z

Rebull at 04:42, 12 July 2012

2012-07-12T04:42:53Z

Rebull at 04:41, 12 July 2012

2012-07-12T04:41:31Z

Rebull at 04:34, 12 July 2012

2012-07-12T04:34:40Z

Rebull at 04:30, 12 July 2012

2012-07-12T04:30:41Z

Rebull at 04:15, 12 July 2012

2012-07-12T04:15:56Z

Rebull at 04:15, 12 July 2012

2012-07-12T04:15:09Z

Rebull at 04:13, 12 July 2012

2012-07-12T04:13:25Z

← Older revision		Revision as of 19:56, 24 July 2012
Line 250:		Line 250:
	*OK. Here was the leading edge of the confusion on source lists. The version from the DVD and the version that I posted on my personal website are a little different, with the latter (more recent) one being better to use. Most of the caveats above still apply.		*OK. Here was the leading edge of the confusion on source lists. The version from the DVD and the version that I posted on my personal website are a little different, with the latter (more recent) one being better to use. Most of the caveats above still apply.
	*On the right is an attempted copy of the stuff I was writing which seemed to help better explain the relationship between these source lists and the questions.		*On the right is an attempted copy of the stuff I was writing which seemed to help better explain the relationship between these source lists and the questions.
		+
		+	'''Updates added 19 July 2012''':
		+	One of my (Luisa's) action items was to look into the 'holes' in the spatial distribution of sources in BRC 27. I overlaid the full wise catalog on the WISE mosaics from Xavier. Of the three images here, there are two that have the whole catalog overplotted -- one is w1 and the other is w4; look at the header of the window in the screen snapshot to see which is which. The third has yellow squares too; see below for that.
		+
		+	<gallery widths="100px" heights="100px" perrow="3">
		+	file:wisecatonwise1.png
		+	file:wisecatonwise4.png
		+	file:wisecat+knownonwise1.png
		+	</gallery>
		+
		+	things to note:
		+
		+	we did a cone search on the wise catalog, and xavier made a square mosaic. so we 'lose' the corners. which is fine; we have a ton of other things to worry about with the objects we have.
		+
		+	there are far more sources seen at w1 that w4 (and you should know why by now!) :) most of the red circles are nicely on sources seen in w1; there are far fewer sources in the w4 image.
		+
		+	if you draw a virtual line across the circle of sources from "11:00" to "4:00", the surface density of sources (e.g., how many sources there are on the screen per chunk of screen) is much higher on the lower left than on the upper right. if you look at the w1 image, you can see that this is for the most part consistent with what you actually see. at "12:00", i really don't see any sources (or many sources) in the black in between sources. it's even doing a pretty good job finding sources on top of the nebulous regions in the image, though some of them are likely to be false sources created by the computer doing the source detection 'seeing' ridges of nebulosity as instead sources.
		+
		+	but the bottom middle is more of a problem. there is a bright source and nebulosity there. and i can see taht it caught a lot of the sources there. but there are a lot of legitimate sources, some reasonably bright, in those holes of "no catalog sources".
		+
		+	THIS MEANS that if there are new legitimate YSOs here, we will not find them because they are not in the catalog. fortunately, not many of them are very bright, otherwise i would be worried. fainter means more likely to be xgal.
		+
		+	THIS ALSO MEANS that if any of our known objects fall in this region, they will appear to not have WISE detections when instead they actually should.
		+
		+	So, i plotted up everything on W1 - red circles is still the wise catalog, and now yellow boxes are the catalog of known YSOs.
		+
		+	there are several yellow boxes that are just outside of our 20 arcmin radius region of interest; i think it's ok to drop those off our list of things to think about.
		+
		+	Most of the yellow boxes also have a red circle underneath. but there are many, especially in the region with no wise sources in the catalog, where i can see something in the image, it's just not in the catalog. there are several of these. i
		+	count 8 or 10 at least. most of these already have IRAC photometry, so we probably have enough to work with them. only a few are going to be problem children. for the purposes of the poster we can just 'punt' and say "these can be seen in the wise image but we don't yet have wise photometry for them." in the long term, we will either have to persuade the wise team to do the photoemtry for them, or we will do aperture photoemtry on the UNCALIBRATED wise image for these targets and ~10-20 others of comparable brightness that ARE in the wise catalog and then bootstrap the calibration and interpolate what the wise photometry of these objects really are.
		+
		+	we will have to make note of any others that are in the 'known yso' list, within 20 arcmin of the center (eg not in the corners), and missing wise photometry, then go look into them.
		+
		+
		+

@@ Line 45: / Line 45: @@
 '''Updates added July 2012''':
 *All of the level 2 (pbcd) data within BRC 27 is actually more than 1.5 GB of data, largely because of several very large mosaics that just brush the eastern edge of the region we care about. During the visit, we decided for those purposes to just download the BRC 27 small pointings (AORKEY 17512192 for IRAC and AORKEY 17512448 for MIPS).
-*[[Using_the_SHA,_long_version#Contents_of_your_download | Cheatsheet for filenames of the files you get in a download]], e.g., which of these files do I want to use? The punchline: *maic.fits are the MosAIC, *munc.fits are the UNCertainties, *mcov.fits are the COVerage. It will sort these into directories by channel and level (e.g., /r17512192/ch1/pbcd/)
+*[[Using_the_SHA,_long_version#Contents_of_your_download | Cheatsheet for filenames of the Spitzer files you get in an SHA download]], e.g., which of these files do I want to use? The punchline: *maic.fits are the MosAIC, *munc.fits are the UNCertainties, *mcov.fits are the COVerage. It will sort these into directories by channel and level (e.g., /r17512192/ch1/pbcd/)
 *You also have copies of these mosaics on the DVD. The copy on the DVD comes from my own reprocessing last year for last year's NITARP team, so the filenames are a little different (as are the mosaics!).
 *Ultimately, you should also download the level 2 (pbcd) mosaics for the other observations that hit the region we care about. On the left is a map of the region with the different footprints indicated; click to get a full-resolution version. Here is a regions file so that you can overlay these footprints on any image in ds9: [[file:brc27footprints.reg.txt]]

← Older revision		Revision as of 04:45, 12 July 2012
Line 427:		Line 427:
	**load the image into ds9. put north up. find the object. note the x and y. type the x and y into APT.		**load the image into ds9. put north up. find the object. note the x and y. type the x and y into APT.
	**under APT's tools, look for an option like "object locator" and type in the coordinates. It should be ##h##m##s and -##d##m##s format. One person could not make this work. Most of the rest seemed to be able to make this work.		**under APT's tools, look for an option like "object locator" and type in the coordinates. It should be ##h##m##s and -##d##m##s format. One person could not make this work. Most of the rest seemed to be able to make this work.
		+	*For making CMDs/CCDs, you need magnitudes. For making SEDs, you need flux densities. Sometimes you get magnitudes from archives, and sometimes you get flux densities. You need to be able to convert fluently between these quantities.
		+	*When comparing multiple teams' measurements of the brightness of an object, it is easier to do if converted to magnitudes first.

@@ Line 424: / Line 424: @@
 and I picked badly for at least 2 or 3 of the 5. The last two on the list are probably the best ones to try, and we will probably agree better on those (once you get to them) than the first three.
 *The image, when loaded into APT, is NOT north-up. There are three ways we came up with to find the object in the image:
 **load the image into ds9. put north up. find the object. note the x and y. type the x and y into APT.
 **under APT's tools, look for an option like "object locator" and type in the coordinates. It should be ##h##m##s and -##d##m##s format. One person could not make this work. Most of the rest seemed to be able to make this work.
@@ Line 431: / Line 432: @@
 =Putting this in context a little=
-This step is important for this particular project, because of the nature of the existing literature for the objects we are studying. Again, we may or may not get to this before you leave California, but it would probably be good to at least try because it will be easier to talk about it in person.
+This step is important for this particular project, because of the nature of the existing literature for the objects we are studying. Again, we may or may not get to this before you leave California.
 '''Big picture goal''': Understand how what we did is different than what others (Chauhan, Choudhury, (Rebull/Johnson)) did with the IRAC (IRAC+MIPS) data to find young stars.

@@ Line 300: / Line 300: @@
 '''Updates added July 2012''':
-*The group accomplished a W1-W2 vs W3-W4 plot. W1-W2 is centered on zero, because most of the objects seen in W1 and W2 are plain stars, so the color is zero. W3-W4 is notably NOT zero, because the only objects seen at W4 are the ones notably bright at W4, so they all are brighter than plain stars at W4. This gave me a minor heart attack until I figured it out.
+*When reading in Xavier's file, it is tab-delimited.
 *Some of the group accomplished a K vs. W1-W4 plot, or a W1 vs. W1-W4 plot. These look like what I expected. The YSO candidates are bright and red, generally.
 *There are other CMDs you can try. See the CG4 paper or the Taurus paper or Xavier's paper (appendix) for ideas.
@@ Line 367: / Line 369: @@
 *''What do you need to get?'' everything into Jy, which are units of Fnu. Then convert your Fnu in Jy into Fnu in cgs units, ergs/s/cm2/Hz, so multiply by 10^-23. Then convert your Fnu into Flambda in cgs units, so multiply by c/lambda^2, with c=2.99d10 cm/s and lambda in cm (not microns!).  Then get lambda*Flambda by multiplying by lambda in cm.  Plot log (lambda*Flambda) vs. log (lambda).
 *I believe that most of you were just trying to get JHK+WISE bands onto an SED. Ultimately, you need to get "all the bands we can" on each SED for each "interesting" object.  We have UBVRI for several literature objects. We have FTN r,i for lots of objects near the middle (and will get more). We have IRAC, MIPS data for SOME of the sources. We can obtain more for SOME of the remaining ones.
-*Once you make your first SED, the rest are easy.
+*Once you make your first SED correctly, the rest are easy.
 *Then you need to look through each of the SEDs and decide which look like you expect, which need photometry to be checked, and which seem unlikely to be legitimate YSOs. This is a judgement call, and your judgement will improve with time, just as it did for the source matching in the Spring.
 *The issue with Chauhan109 above is that there is a bright nebulous patch in the WISE images that either is being inappropriately tagged as a point source (with its flux densities attached to this source) or whose brightness is contaminating the photometry beyond recovery. The Spitzer data is critical for sorting out what is going on here. This will not be the first time we are grateful for Spitzer data. (And I still don't know what is going on with the optical data - it's wrong for this source, but this is the best possible match given the information we have in the literature.
@@ Line 412: / Line 414: @@
 #You can repeat this on the optical data, but Russ has helpfully reduced the data we have so far. Obtain the full catalog, identify the source matches, and add those points to their corresponding SEDs. CAUTION: these are AB mags, not Vega mags. Does the addition of the optical points change your opinion on which YSO candidates are going to survive?
 #Look at the new photometry with a critical eye. Which points look like they should be double-checked?
 =Putting this in context a little=

@@ Line 298: / Line 298: @@
 #Are the literature sources tagged as YSO candidates by Xavier or not? You may wish to add them with another color/shape symbol. I DO NOT BELIEVE WE GOT TO THIS.
 #''Optional'' next step (may help with larger concepts, or doing this yourself from scratch) - [[Taurus catalog]] has a catalog of legitimate young stars. Where do these objects fall with respect to either the Gutermuth or Koenig colors? Which ones would be retrieved or lost by these color selections? Do the YSO candidates that Xavier sent us have colors like (most of) these YSOs? (Hint: Make a color-color or color-magnitude plot of the entire distribution in the catalog, highlight the YSO candidates with one color symbols, and add the Taurus YSOs to the plots using a different color symbol.)
 =Investigating the images of the objects=

@@ Line 265: / Line 265: @@
 '''For completeness''': The WISE catalog provides some matches to 2MASS, but it is unclear to me exactly where those matches come from, like which sources out of the 2MASS catalog did they use (which error flags did they keep, or did they just look for any matches in the 2MASS catalog?). When I take the WISE catalog's 2MASS matches and compare it to the 2MASS catalog I pull out of Gator myself, I find about 3% fewer matches than the WISE catalog does. I use a matching tolerance of 1 arcsec to match up sources. These 'extra' 3% must be either objects matched to the WISE source that are more than 1 arcsec away, or with a different set of error flags than I would use coming out of 2MASS. What this means in practice is that: (a) we can keep and use the WISE catalog's matches to the 2MASS sources; (b) we should remember this caveat, and if we notice any 2MASS data downstream from here that is weird (like a mismatch in the SED), then we should go back and check those 2MASS matches by hand with the original 2MASS catalog.
 =Making color-color and color-magnitude plots=
-<font color="red">(essentially DONE for brc27, indefinitely pending for other two)</font>
+<font color="red">(toe-dipped for brc27, indefinitely pending for other two)</font>
 OK, fair warning, some math involved, and the start of ''programming'' spreadsheets!
@@ Line 284: / Line 295: @@
 '''Questions for you''':
 #Pick at least one color-color or color-magnitude plot to make.  Figure out a way to ignore the "no data" flags (exactly what they are depends on which file you are starting from). Does the photometry seem ok? Where are the plain stars?  Where are the IR excess objects? START FROM A COMPLETE WISE CATALOG FOR STARTERS.
-#Which objects are selected by Xavier's method as YSO candidates? You may wish to overplot them with a different color/shape symbol. You may wish to try drawing the line segments on the plot too. OVERPLOT ONE OF THE OTHER CATALOGS ON THE SAME PLOT.
+#Which objects are selected by Xavier's method as YSO candidates? You may wish to overplot them with a different color/shape symbol. You may wish to try drawing the line segments on the plot too. OVERPLOT XAVIER'S YSOS ON THE SAME PLOT. Note not Xavier's entire catalog, but just the YSO candidates.
-#Are the literature sources tagged as YSO candidates by Xavier or not? You may wish to add them with another color/shape symbol.
+#Are the literature sources tagged as YSO candidates by Xavier or not? You may wish to add them with another color/shape symbol. I DO NOT BELIEVE WE GOT TO THIS.
-#Optional next step (may help with larger concepts, or doing this yourself from scratch) - [[Taurus catalog]] has a catalog of legitimate young stars. Where do these objects fall with respect to either the Gutermuth or Koenig colors? Which ones would be retrieved or lost by these color selections? Do the YSO candidates that Xavier sent us have colors like (most of) these YSOs? (Hint: Make a color-color or color-magnitude plot of the entire distribution in the catalog, highlight the YSO candidates with one color symbols, and add the Taurus YSOs to the plots using a different color symbol.)
+#''Optional'' next step (may help with larger concepts, or doing this yourself from scratch) - [[Taurus catalog]] has a catalog of legitimate young stars. Where do these objects fall with respect to either the Gutermuth or Koenig colors? Which ones would be retrieved or lost by these color selections? Do the YSO candidates that Xavier sent us have colors like (most of) these YSOs? (Hint: Make a color-color or color-magnitude plot of the entire distribution in the catalog, highlight the YSO candidates with one color symbols, and add the Taurus YSOs to the plots using a different color symbol.)
 =Investigating the images of the objects=
-<font color="red">pending for all of them - put off because can do training for this over the phone</font>
+<font color="red">July 2012: pending for all of them - put off because can do training for this over the phone</font>
 ''This will take some time to complete, and we may not be able to finish it for any one cloud while you are in California.''
@@ Line 338: / Line 349: @@
 Pick some objects to plot up, maybe some of the previously-identified ones from above would be a good place to start, or the ones you flagged above as having an IR excess. Start with just one. You will ultimately plot log (lambda*F(lambda)) vs log (lambda) -- see the [[Units]] page. It will take time to get the units right, but once you do it right the first time, all the rest come along for free (if you're working in a spreadsheet). Spend some time looking at the SEDs. Look at their similarities and differences. Identify the bad ones, and discuss with the others why/whether to drop them off the list of YSO candidates.  See also stuff above about data at other wavelengths, and include literature/archival data from other sources where appropriate and possible. Make sure to keep careful track of those things that are limits rather than detections.
 '''Questions for you''':
@@ Line 349: / Line 356: @@
 #What's the deal with this one (why does it look like this)? (In my SED, the y-axis units are cgs units [sorry], *=FTN data, +=optical literature data, diamonds=2mass, circles=irac, stars=WISE, arrows=limits, and boxes=MIPS if they exist, which they don't here.)
 [[file:chauhan109sed.png]]
 =Doing Spitzer photometry=

@@ Line 239: / Line 239: @@
 #For each of the known YSOs, you have the RA/Dec - find the same objects in the WISE ''catalog''. Which objects are the matches? What constitutes a 'match'? Are there any with no matches?
 Now, start to pull together stuff from this step plus those above. Here are plots (click to get a big one) of the spatial distribution of sources in all three regions we care about, with the symbols as defined in the plots (black dot = object in catalog, meaning WISE+literature data+my Spitzer catalog from last year's group; red circle = identified in the literature as a YSO; blue x = identified by Xavier's automatic code as likely clump of nebulosity; green diamond = identified by Xavier's automatic code as possible YSO; yellow diamond = IRAC data available in the catalog). Why are there some regions where WISE sources are missing? Why are the Spitzer sources where they are? (Is this all the Spitzer sources that are possible, or not? Why?) Why is there a patch of black in BRC 38 (hint: Beltran et al.)?
 <gallery widths="100px" heights="100px" perrow="3">
 file:brc27where.png
@@ Line 247: / Line 246: @@
 '''Updates added July 2012''':
 *OK. Here was the leading edge of the confusion on source lists. The version from the DVD and the version that I posted on my personal website are a little different, with the latter (more recent) one being better to use. Most of the caveats above still apply.
-*Here is an attempted copy of the stuff I was writing which seemed to help better explain the relationship between these source lists and the questions.
+*On the right is an attempted copy of the stuff I was writing which seemed to help better explain the relationship between these source lists and the questions.

@@ Line 234: / Line 234: @@
 ##CAUTION 3: I did NOT do a good job documenting these files. Shame on me. Should have a date (missing), and should explain the columns. "rebullwhyhere" is the note from last year's BRC paper as to why we were caring about it -- was it a previously known source, did it have an IR excess? The column "yso" is =1 if it is tagged as a yso; everything in the "justysos" files do, but not everything in the full files have yso=1.  See also next item...
 ##CAUTION 4, AND THIS ONE'S A BIGGIE: '''This file has a mixture of detections and limits, measurements and errors, flux densities and magnitudes (with two different definitions of the latter!).''' The UBVRI are all Vega mags, no limits, and only some have errors. The Sloan r and i bands do have some limits, and these are AB mags. The JHK are all Vega mags, and some have limits. The IRAC and MIPS data are all in FLUX DENSITIES (uJy), and often have limits.  Just to add to the confusion (sorry!!!) the limits for this entire file are ALL IN FLUX SPACE, e.g., the sloan r mag for 07040072-1123234 is a limit, listed as 20.60, with a < (less than) sign; what this really means is "the true value of this object in this band is somewhere fainter than this value." For mags, this should be a GREATER number. So I just screwed up, but since I already burned all the DVDs, it's probably easier to explain it than fix it at this point.
 #For each of the known objects, you have the RA/Dec - what is the easiest way to find the objects in the images you obtained above? (Hint: you want a ds9 regions file: [[file:brc27knownysos.reg.txt]] [[file:ic1396knownysos.reg.txt]])  Take notes on any that look like the photometry may be corrupted or that you can easily dismiss as galaxies or other contaminants now that you are looking at better and/or different data than they had before.
 #Look at the files from WISE and from Xavier. The files from WISE are better documented in the top of the file than mine are; Xavier's files have documentation in a separate file.  How is the information presented in the WISE catalogs and Xavier's catalogs the same/different? What information do we need?
@@ Line 242: / Line 239: @@
 #For each of the known YSOs, you have the RA/Dec - find the same objects in the WISE ''catalog''. Which objects are the matches? What constitutes a 'match'? Are there any with no matches?
 Now, start to pull together stuff from this step plus those above. Here are plots (click to get a big one) of the spatial distribution of sources in all three regions we care about, with the symbols as defined in the plots (black dot = object in catalog, meaning WISE+literature data+my Spitzer catalog from last year's group; red circle = identified in the literature as a YSO; blue x = identified by Xavier's automatic code as likely clump of nebulosity; green diamond = identified by Xavier's automatic code as possible YSO; yellow diamond = IRAC data available in the catalog). Why are there some regions where WISE sources are missing? Why are the Spitzer sources where they are? (Is this all the Spitzer sources that are possible, or not? Why?) Why is there a patch of black in BRC 38 (hint: Beltran et al.)?
 '''Updates added July 2012''':