Difference between revisions of "How can I find already-reduced Spitzer data?"

From CoolWiki
Jump to navigationJump to search
m
 
(15 intermediate revisions by the same user not shown)
Line 1: Line 1:
 +
There are, or will be, two different kinds of already-reduced Spitzer data that are (or will be) available.  (1) In late 2011-2012, the SSC will be releasing fully processed mosaics and source lists. This will be a tremendous resource, but at the same time, will not wring everything you can get out of the data. The source lists, for example, will pull off the brightest sources. Right now (May 2011), these data don't even fully exist, so they are not available yet anywhere, but they will be available via the SHA.  (2) Some Spitzer observer teams delivered products back to the SSC for use by the community.  All of this stuff (at least, that stuff that we've received and ingested) is actually available now through the SHA, and that's what the rest of this page is about. Read on!
 +
 +
'''What is a Legacy Program?'''
 
Spitzer started a completely new kind of observing program, called Legacy programs.  These are ''large, coherent data sets (organized around a scientific theme)'', and the observers agreed to waive their proprietary rights, e.g., they let their ''Spitzer data go public immediately.''  If that weren't enough, they also ''deliver back to the SSC so-called enhanced data products'', which are, at minimum, reduced Spitzer data, but often include observations at other wavelengths as well.
 
Spitzer started a completely new kind of observing program, called Legacy programs.  These are ''large, coherent data sets (organized around a scientific theme)'', and the observers agreed to waive their proprietary rights, e.g., they let their ''Spitzer data go public immediately.''  If that weren't enough, they also ''deliver back to the SSC so-called enhanced data products'', which are, at minimum, reduced Spitzer data, but often include observations at other wavelengths as well.
  
You can read all about the [http://ssc.spitzer.caltech.edu/spitzermission/observingprograms/legacy/ Spitzer Legacy program] on the SSC website, where you can find a list of all the Legacy programs.  You can see what a tremendous variety of programs there are.  Each program has a plan of what they will deliver back to the SSC and when - that can be obtained by clicking on the blue dot on the far right of that page, one per Legacy program.  The oldest Legacy programs have had the longest time to work on their data, so they are the ones that have delivered the most data back to the SSC.  The newest programs in many cases haven't even obtained their data yet.
+
You can read all about the [http://irsa.ipac.caltech.edu/data/SPITZER/docs/spitzermission/observingprograms/legacy/ Spitzer Legacy program] elsewhere on the web, where you can find a list of all the Legacy programs.  You can see what a tremendous variety of programs there are.  Each program has a plan of what they will deliver back to the SSC and when - that can be obtained by clicking on the blue dot on the far right of that page, one per Legacy program.  The oldest Legacy programs have had the longest time to work on their data, so they are the ones that have delivered the most data back to the SSC.  The newest programs in many cases haven't even obtained their data yet.
  
 
'''Why would you want to use data from these projects?'''   
 
'''Why would you want to use data from these projects?'''   
Line 9: Line 12:
  
 
'''Which objects are in these projects?'''   
 
'''Which objects are in these projects?'''   
These projects do not cover the whole sky.  They're not even everything that Spitzer has ever observed.  But they are a lot of data anyway.  If the object in which you are interested is lucky enough to be in one of these regions, then by all means, take  advantage of it!  There is no one master list of all of the things that are just in Legacy projects. You kind of have to poke around on your own to see if your object is in one of these regionsIf you [[How_can_I_find_any_prior_observations_for_an_object%3F| use Leopard to search for prior observations]], and one of the programs that is returned by your search is a Legacy program, e.g., is listed [http://ssc.spitzer.caltech.edu/spitzermission/observingprograms/legacy/ on the complete list of Legacy projects], then probably you will be able to get data products from the Legacy teams for your objectRemember, though, it takes time for the teams to chew through the data, so it's possible that the data have been taken but not yet returned as enhanced products to the SSC. 
+
These projects do not cover the whole sky.  They're not even everything that Spitzer has ever observed.  But they are a lot of data anyway.  If the object in which you are interested is lucky enough to be in one of these regions, then by all means, take  advantage of it!  New in the most recent version of the SHA, there is a way to search all of these projects at once. Remember, though, it takes time for IRSA to ingest the products, so some products that have been delivered to the SSC are not yet available through the SHA. However, the vast majority are searchable via the SHA.  
 +
 
 +
 
 +
=Example: getting images of M51=
 +
 
 +
Go to the SHA, and do a position search. Enter "M51". Be sure to select the "enhanced products" tab.
 +
 
 +
[[image:m51searchstart.png]]
 +
 
 +
Click search. The results are shown here:
 +
 
 +
[[image:m51searchresults.png]]
 +
 
 +
What this is telling you is that two projects, SINGS and LVL, observed (or at least delivered) images of M51. There are 134 images from SINGS that are relevant to M51, and 32 images from LVL.  There are also 104 spectra from SINGS taken with IRS and 2 taken with MIPS-SEDTo learn more about the specific data delivery, you can click on the "more" link, and be taken to the documents delivered by the teams. To see what images are available, click on the "SINGS" or "LVL" link on the left. You then see this:
 +
 
 +
[[image:m51singsresults.png]]
 +
 
 +
The list on the left is each of the images. The colored polygons on the right show you the area covered by each of those images. The blue highlighted polygon on the right corresponds to the green highlighted row on the left. This works both ways; you can pick a row and the blue highlight will change, or you can pick a polygon and the green highlight will change. Note that it is only showing you the first 50 of the possible 134 images. The filenames of the images tell you more about what the image is, or at least gives you a hint. You may need to go back to the delivery documents to find out exactly what it is.  In this list, we see MIPS SED image cubes, IRS Ne III emission line images, IRAC and MIPS images, Halpha images, V and R images, and even more. Tons of resources here, and you will need to explore them to find out which you want.  
 +
 
 +
Just like with the rest of the SHA, to download them, click on the checkbox on the far left and then click on "prepare download" (button at the top of that column) to package up the files. You can then download them via the Background Monitor.
 +
 
 +
If you go to the [http://ssc.spitzer.caltech.edu/spitzermission/observingprograms/legacy/ list of Legacy projects], you can find the [http://sings.stsci.edu/ SINGS team website], and an [http://adsabs.harvard.edu/abs/2003PASP..115..928K overview paper about SINGS].
 +
 
 +
=Example: getting images and photomety of M16=
 +
 
 +
Go to the SHA, and pick Inventory Search. Enter "M16". Click searchThe results then look like this; it may take a few seconds for it to come back with the catalog results:
  
 +
[[image:m16inventoryresults.png]]
  
=Example: getting images of M51 from the SINGS program=
+
What this is telling you is that two projects, GLIMPSE and MIPSGAL, observed this region, and both delivered images. Only GLIMPSE delivered a catalog back, but two flavors of catalogs to choose from.  There is 1 image from MIPSGAL, 16 images from GLIMPSE, and results from two of GLIMPSE's catalogs.
  
The SINGS program observed a long list of nearby galaxies with IRAC and MIPS.  If you go to the [http://ssc.spitzer.caltech.edu/spitzermission/observingprograms/legacy/ list of Legacy projects], you can find the [http://sings.stsci.edu/ SINGS team website], an [http://adsabs.harvard.edu/abs/2003PASP..115..928K overview paper about SINGS], and a [http://ssc.spitzer.caltech.edu/spitzermission/observingprograms/legacy/sings/ list of their data deliveries]. This is one of the oldest projects, so they have had five deliveries, including their final delivery.  They deliver images (in Spitzer and optical wavelengths) and Spitzer spectra.  Deliveries that the teams make to us get ingested into IRSA, which will be the data's permanent home after the SSC shuts its doors for good. You can either access the most recent version of the data directly by clicking on "direct data access," or use IRSA to search their deliveries.
+
As above, in order to learn more about the specific data delivery (what does the stuff about "complete/reliable" mean?), you can click on the links just to the right of each entry, and be taken to the documents delivered by the teams. To see a listing of what images are available, click on the "GLIMPSE" or "MIPSGAL" link on the left. For GLIMPSE, you then see this:
  
If you'd like images of M51, the Whirlpool galaxy, go to "direct data access" and find the [http://data.spitzer.caltech.edu/popular/sings/20070410_enhanced_v1/ngc5194/ directory for NGC 5194] (which is a synonym for M51).  In that directory, you'll find a directory for IRAC, MIPS, IRS, and "Ancillary".  Each of these directories contains fits images (final mosaics) of this object in those wavelengths.  The "Ancillary" directory includes BVRI and Halpha images.
+
[[image:m16imageresults.png]]
  
These data are all free for your use.
+
As above, the list on the left is each of the images. The colored polygons on the right show you the area covered by each of those images. The blue highlighted polygon on the right corresponds to the green highlighted row on the left. This works both ways; you can pick a row and the blue highlight will change, or you can pick a polygon and the green highlight will change. The filenames of the images tell you more about what the image is, or at least gives you a hint. You may need to go back to the delivery documents to find out exactly what it is.  
  
=Example: getting images and photomety of M16 from the GLIMPSE program=
+
Just like with the rest of the SHA, to download them, click on the checkbox on the far left and then click on "prepare download" (button at the top of that column) to package up the files. You can then download them via the Background Monitor.
  
The GLIMPSE program is actually three generations of programs that surveyed/is surveying a large portion of the galactic plane with IRAC. (NB: it has a sister program, MIPSGAL, which is doing the same region with MIPS.) If you go to the [http://ssc.spitzer.caltech.edu/spitzermission/observingprograms/legacy/ list of Legacy projects], you can find the [http://www.astro.wisc.edu/sirtf/ GLIMPSE team website], an [http://adsabs.harvard.edu/abs/2003PASP..115..953B overview paper about GLIMPSE], and a [http://ssc.spitzer.caltech.edu/spitzermission/observingprograms/legacy/glimpse/ list of their data deliveries], where it becomes apparent that there are three generations of this program all listed together on one page. The original GLIMPSE program is one of the oldest projects, so they have had many deliveries.  They deliver images and catalogs. They have delivered so much data that, unlike SINGS above, this time you really do want to search via IRSA because otherwise it's too hard to find what you want.
+
To investigate the catalogs, go back to the "Summarized View" tab and click on "GLIMPSE" next to the catalog you want to explore. It may take several seconds (or longer, depending on your search radius) for it to go get the catalog. It will probably spin off to the background monitor (upper right of your screen). It might display the catalog right away, or you may have to pick "show catalog" from the background monitor. It should look like this:
  
M16, the Eagle Nebula, is in the region covered by GLIMPSE.  If you'd like images of M16, click on "[http://irsa.ipac.caltech.edu/data/SPITZER/GLIMPSE/ IRSA search]" and search on M16.  The search will return several "postage stamp" image cutouts of the region, at different resolutions.  Go down to the bottom of the returned page to download them.
+
[[image:m16catalogresults.png]]
  
If you'd like source lists (extracted photometry) from M16, you need to use Gator to [http://irsa.ipac.caltech.edu/cgi-bin/Gator/nph-dd?catalog=glimpse_s05|search the GLIMPSE catalog] -- this works just like searching the 2MASS catalog using Gator if you've done that before.  Type "M16" in as the object name, and whatever search radius you would like, and it will come back with a list of objects in that region and fluxes at all four IRAC bands, along with a bunch of other information (like quality flags), just like 2MASS.  For any given region (e.g., your own region, not necessarily M16), you may need to try separately searching each GLIMPSE delivery.  As of this writing, they are not (yet?) combined all together into one master catalogYou may need to make a decision about whether you want to search the catalog that is "highly reliable" (e.g., just the brighter things, but all items listed in the catalog are likely to be real sources, not artifacts), or "more complete" (e.g., goes fainter, but may include some objects listed as real sources but that are artifacts if you look at the images).  Which you pick depends on what kind of science you are doing.
+
Each of those columns are those that were delivered by the team, and it varies from program to program. If it is not apparent from the column headings, you will need to go back to the delivery documentation to figure out what it is, but generally speaking, everything should have a name, ra, dec, and some measurement of the brightness (flux density or magnitudes) in at least one bandThe 50 sources currently displayed on the left might be shown on an image on the right. As with the images, it should work such that the blue highlighted source on the right corresponds to the green highlighted row on the left, and this works both ways (e.g., pick a row in the table or pick a source on the image). To show more sources, increase the page size, but be advised that very long tables take a long time to render!
  
These data are all free for your use.
+
If you go to the [http://ssc.spitzer.caltech.edu/spitzermission/observingprograms/legacy/ list of Legacy projects], you can find the [http://www.astro.wisc.edu/sirtf/ GLIMPSE team website], and an [http://adsabs.harvard.edu/abs/2003PASP..115..953B overview paper about GLIMPSE].  Look there for MIPSGAL stuff too.

Latest revision as of 16:24, 10 May 2012

There are, or will be, two different kinds of already-reduced Spitzer data that are (or will be) available. (1) In late 2011-2012, the SSC will be releasing fully processed mosaics and source lists. This will be a tremendous resource, but at the same time, will not wring everything you can get out of the data. The source lists, for example, will pull off the brightest sources. Right now (May 2011), these data don't even fully exist, so they are not available yet anywhere, but they will be available via the SHA. (2) Some Spitzer observer teams delivered products back to the SSC for use by the community. All of this stuff (at least, that stuff that we've received and ingested) is actually available now through the SHA, and that's what the rest of this page is about. Read on!

What is a Legacy Program? Spitzer started a completely new kind of observing program, called Legacy programs. These are large, coherent data sets (organized around a scientific theme), and the observers agreed to waive their proprietary rights, e.g., they let their Spitzer data go public immediately. If that weren't enough, they also deliver back to the SSC so-called enhanced data products, which are, at minimum, reduced Spitzer data, but often include observations at other wavelengths as well.

You can read all about the Spitzer Legacy program elsewhere on the web, where you can find a list of all the Legacy programs. You can see what a tremendous variety of programs there are. Each program has a plan of what they will deliver back to the SSC and when - that can be obtained by clicking on the blue dot on the far right of that page, one per Legacy program. The oldest Legacy programs have had the longest time to work on their data, so they are the ones that have delivered the most data back to the SSC. The newest programs in many cases haven't even obtained their data yet.

Why would you want to use data from these projects? (a) They have used observing strategies that in many cases broke their observations over several AORs, and they have already done the work of combining multiple AORs where necessary. They have used their expertise to create the best possible mosaic from these data; (b) They have (in many cases) already done the photometry for you as well, so you can just jump in and get the fluxes directly from them right away; (c) I guarantee you that the original astronomers who designed these observations and created these mosaics and photometry have NOT had time (and probably never will have time) to extract every little bit of information out of them. There is a LOT of science just waiting to be discovered in these data!

Which objects are in these projects? These projects do not cover the whole sky. They're not even everything that Spitzer has ever observed. But they are a lot of data anyway. If the object in which you are interested is lucky enough to be in one of these regions, then by all means, take advantage of it! New in the most recent version of the SHA, there is a way to search all of these projects at once. Remember, though, it takes time for IRSA to ingest the products, so some products that have been delivered to the SSC are not yet available through the SHA. However, the vast majority are searchable via the SHA.


Example: getting images of M51

Go to the SHA, and do a position search. Enter "M51". Be sure to select the "enhanced products" tab.

M51searchstart.png

Click search. The results are shown here:

M51searchresults.png

What this is telling you is that two projects, SINGS and LVL, observed (or at least delivered) images of M51. There are 134 images from SINGS that are relevant to M51, and 32 images from LVL. There are also 104 spectra from SINGS taken with IRS and 2 taken with MIPS-SED. To learn more about the specific data delivery, you can click on the "more" link, and be taken to the documents delivered by the teams. To see what images are available, click on the "SINGS" or "LVL" link on the left. You then see this:

M51singsresults.png

The list on the left is each of the images. The colored polygons on the right show you the area covered by each of those images. The blue highlighted polygon on the right corresponds to the green highlighted row on the left. This works both ways; you can pick a row and the blue highlight will change, or you can pick a polygon and the green highlight will change. Note that it is only showing you the first 50 of the possible 134 images. The filenames of the images tell you more about what the image is, or at least gives you a hint. You may need to go back to the delivery documents to find out exactly what it is. In this list, we see MIPS SED image cubes, IRS Ne III emission line images, IRAC and MIPS images, Halpha images, V and R images, and even more. Tons of resources here, and you will need to explore them to find out which you want.

Just like with the rest of the SHA, to download them, click on the checkbox on the far left and then click on "prepare download" (button at the top of that column) to package up the files. You can then download them via the Background Monitor.

If you go to the list of Legacy projects, you can find the SINGS team website, and an overview paper about SINGS.

Example: getting images and photomety of M16

Go to the SHA, and pick Inventory Search. Enter "M16". Click search. The results then look like this; it may take a few seconds for it to come back with the catalog results:

M16inventoryresults.png

What this is telling you is that two projects, GLIMPSE and MIPSGAL, observed this region, and both delivered images. Only GLIMPSE delivered a catalog back, but two flavors of catalogs to choose from. There is 1 image from MIPSGAL, 16 images from GLIMPSE, and results from two of GLIMPSE's catalogs.

As above, in order to learn more about the specific data delivery (what does the stuff about "complete/reliable" mean?), you can click on the links just to the right of each entry, and be taken to the documents delivered by the teams. To see a listing of what images are available, click on the "GLIMPSE" or "MIPSGAL" link on the left. For GLIMPSE, you then see this:

M16imageresults.png

As above, the list on the left is each of the images. The colored polygons on the right show you the area covered by each of those images. The blue highlighted polygon on the right corresponds to the green highlighted row on the left. This works both ways; you can pick a row and the blue highlight will change, or you can pick a polygon and the green highlight will change. The filenames of the images tell you more about what the image is, or at least gives you a hint. You may need to go back to the delivery documents to find out exactly what it is.

Just like with the rest of the SHA, to download them, click on the checkbox on the far left and then click on "prepare download" (button at the top of that column) to package up the files. You can then download them via the Background Monitor.

To investigate the catalogs, go back to the "Summarized View" tab and click on "GLIMPSE" next to the catalog you want to explore. It may take several seconds (or longer, depending on your search radius) for it to go get the catalog. It will probably spin off to the background monitor (upper right of your screen). It might display the catalog right away, or you may have to pick "show catalog" from the background monitor. It should look like this:

M16catalogresults.png

Each of those columns are those that were delivered by the team, and it varies from program to program. If it is not apparent from the column headings, you will need to go back to the delivery documentation to figure out what it is, but generally speaking, everything should have a name, ra, dec, and some measurement of the brightness (flux density or magnitudes) in at least one band. The 50 sources currently displayed on the left might be shown on an image on the right. As with the images, it should work such that the blue highlighted source on the right corresponds to the green highlighted row on the left, and this works both ways (e.g., pick a row in the table or pick a source on the image). To show more sources, increase the page size, but be advised that very long tables take a long time to render!

If you go to the list of Legacy projects, you can find the GLIMPSE team website, and an overview paper about GLIMPSE. Look there for MIPSGAL stuff too.