Difference between revisions of "How do I download data from Spitzer?"
m |
|||
| (26 intermediate revisions by the same user not shown) | |||
| Line 5: | Line 5: | ||
==Introduction== | ==Introduction== | ||
| − | The Spitzer Heritage Archive (SHA) is the permanent home for all of the data collected during the Spitzer mission, plus all the documentation you need to understand it all. The SHA | + | The Spitzer Heritage Archive (SHA) is the permanent home for all of the data collected during the Spitzer mission, plus all the documentation you need to understand it all. The SHA is formally part of IRSA's archive holdings (no longer 'owned' by the SSC). The SHA provides a web-based interface to the Spitzer archive, and it lives here: http://irsa.ipac.caltech.edu/applications/Spitzer/SHA/ |
| + | Because it is web-based, you do not need to download and install software that is platform-dependent. It should "just work" in whatever browser you use (though, for really new or really old browsers, your mileage may vary)! There is online help for the SHA -- see the help menu in the upper right (of the red menu bar). There are also several other ways to get help; see [http://irsa.ipac.caltech.edu/data/SPITZER/docs/spitzerdataarchives/ here] -- look under "Spitzer Heritage Archive Documentation". The cookbook's first few chapters has detailed step-by-step recipes (one of which was originally developed for a NITARP team), the User's Guide is a standalone PDF manual, and some instructional videos are linked in as both YouTube and Flash copies. | ||
| − | + | The software that used to be the primary mechanism for pulling data from the archive is called Leopard. There might be some lingering references to Leopard on the wiki, though we have tried to clean them all out. | |
| − | |||
| − | |||
| − | The software that used to be the primary mechanism for pulling data from the archive is called Leopard. Leopard | ||
== Basic Terminology == | == Basic Terminology == | ||
| − | |||
| − | |||
An individual Spitzer observation sequence is an AOR, or Astronomical Observation Request. In certain cases (often calibration or sometimes science observations), you may also see an IER, or Instrument Engineering Request. Either one involves many individual frames, as well as observer name, date of observation, object or area of the sky observed, and instrument used (IRAC, MIPS, or IRS)-- these are all part of the AOR. All of Spitzer's operations (planning, scheduling, processing) have been centered around these units (AORs or IERs). Now, for the SHA, we are starting to move away from that, but there are some things that are still only available on an AOR basis, so we really can't escape them. | An individual Spitzer observation sequence is an AOR, or Astronomical Observation Request. In certain cases (often calibration or sometimes science observations), you may also see an IER, or Instrument Engineering Request. Either one involves many individual frames, as well as observer name, date of observation, object or area of the sky observed, and instrument used (IRAC, MIPS, or IRS)-- these are all part of the AOR. All of Spitzer's operations (planning, scheduling, processing) have been centered around these units (AORs or IERs). Now, for the SHA, we are starting to move away from that, but there are some things that are still only available on an AOR basis, so we really can't escape them. | ||
| Line 22: | Line 18: | ||
Raw data that are fundamentally unprocessed are "Level 0" data. As NITARP folks, you should never encounter (or want to encounter, really) Level 0 data. | Raw data that are fundamentally unprocessed are "Level 0" data. As NITARP folks, you should never encounter (or want to encounter, really) Level 0 data. | ||
| − | The individual data frames that emerge, calibrated, from the Spitzer pipeline are "Level 1," or "Basic Calibrated Data," or "BCDs." | + | The individual data frames that emerge, calibrated, from the Spitzer pipeline are "Level 1," or "Basic Calibrated Data," or "BCDs." You can get just the BCDs from a region that you want; you don't have to download the whole AOR if it covers a much larger region than you want. As NITARP folks, you probably don't need these Level 1 data. But you might. |
| − | The products that come from combining these individual data frames (such as mosaics) are "Level 2," or "post-BCD," or "PBCD data." These still exist fundamentally on an AOR level, e.g., you can't get a Level 2 mosaic that is just a portion of an AOR. As NITARP folks, you probably want these data. | + | The products that come from combining these individual data frames (such as mosaics) are "Level 2," or "post-BCD," or "PBCD data." These still exist fundamentally on an AOR level, e.g., you can't get a Level 2 mosaic that is just a portion of an AOR. '''As NITARP folks, you probably want these data.''' |
| − | + | You can also get some higher-level processed products (which you might call "Level 3" data, but which in this context are called "Enhanced Products") through this interface. These products are supplemental data that are produced either by the SSC or donated to us by professional astronomers, and represent additional processing. For example, you can get a mosaic combining data from 7 AORs into one big mosaic, with customized (as opposed to hands-off pipeline) processing of image artifacts. Most of the enhanced products in the SHA are delivered by Legacy teams, or developed by the SSC itself. See below for more on this. | |
| − | All of the images come in FITS format. ''(Wondering [[FITS_format|what is FITS format?]])'' (If you are really savvy, you might also care that they are mostly single-plane FITS files. Some | + | All of the images come in FITS format. ''(Wondering [[FITS_format|what is FITS format?]])'' (If you are really savvy, you might also care that they are mostly single-plane FITS files. Some enhanced products will be/are multi-plane FITS.) The other format for some data is IPAC table files (.tbl extension). [http://irsa.ipac.caltech.edu/applications/DDGEN/Doc/ipac_tbl.html IPAC table format] is really just plain text, with a special header. Once you get a file like this, just about anything (including Excel) can read it. ([http://www.youtube.com/watch?v=nCJ3ctOGvNk YouTube video on tbl files, how to access them, and how to get them into Excel (10min)].) |
| − | =Downloading Data: | + | =Downloading Data: Using the SHA, short versions= |
| − | |||
| − | + | *Option 1 : [[SHA Quick Start for NGC 4051]] or [[How can I quickly get a mosaic of my object?]] Both are quick, text only quick start guides. | |
| − | |||
| − | + | *Option 2 : Please see the [http://irsa.ipac.caltech.edu/data/SPITZER/docs/dataanalysistools/cookbook/5/#_Toc288477465 first recipe in the Spitzer Data Analysis Cookbook] (direct link ought to work!). This has screen snapshots. (Developed for professional astronomers. Hopefully it makes sense to you too. Let Luisa know if it doesn't.) | |
| − | + | *Option 3 : Or, see [http://www.youtube.com/watch?v=H0_Jfxxfzkk YouTube QuickStart video] (7.5 min). (Developed for professional astronomers. Hopefully it makes sense to you too. Let Luisa know if it doesn't.) | |
| − | =Downloading Data: [[ | + | =Downloading Data: [[Using the SHA, long version]]= |
| − | + | In more detail! | |
| − | + | *Wiki page: [[Using the SHA, long version]] | |
| − | + | *Also see [http://www.youtube.com/watch?v=VrKtlCwmZSM YouTube video, the long version] (13.5 min). | |
| − | + | (The wiki page was developed for you; the video was developed for professional astronomers.) | |
| − | |||
| − | |||
| + | =Downloading Data: Using the SHA, a concrete example, very long version = | ||
| + | Originally developed especially for the 2010 CG4 team, but then turned into a [http://irsa.ipac.caltech.edu/data/SPITZER/docs/dataanalysistools/cookbook/6/#_Toc288477466 formal chapter] for the professional astronomer's Data Reduction Cookbook. This demo covers the following tasks: Use the SHA to search the Spitzer Archive for all possible and relevant CG4 observations. Use the SHA to assess which of several different observations of the same object will most quickly yield an image that you want. Select data for download, and do it. | ||
| + | [http://irsa.ipac.caltech.edu/data/SPITZER/docs/dataanalysistools/cookbook/6/#_Toc288477466 Recipe 2 from the Cookbook] (direct link should hopefully work!!) | ||
| + | =Downloading Data: [[How can I find already-reduced Spitzer data?]]= | ||
| − | + | The SHA also includes polished mosaics and source lists, with more to come! | |
| − | + | =Downloading Data: [[Using the SHA-searching for a list of objects]]= | |
| − | |||
| − | |||
| − | |||
| − | + | How to search for a large list of objects efficiently. | |
| − | |||
| − | |||
| − | |||
| − | |||
=Downloading Data: [[How can I quickly get a mosaic of my object?]]= | =Downloading Data: [[How can I quickly get a mosaic of my object?]]= | ||
Get me a mosaic, quick! Don't bother me with preambles or complete explanations, I just want a picture. (Also see [[What is a mosaic and why should I care?]]) | Get me a mosaic, quick! Don't bother me with preambles or complete explanations, I just want a picture. (Also see [[What is a mosaic and why should I care?]]) | ||
| − | = | + | =Questions to think about and things to try with the SHA= |
| + | Pick an object to search on, anything you want. | ||
| + | * How many observations are available? Which are imaging? At which bands? | ||
| + | * Can you find any already-processed Spitzer data on this object? | ||
| − | |||
| − | = | + | =[[I'm ready to advance to a highly technical and in-depth discussion on processing Spitzer data.]]= |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
This tells you how to start from the same place professional astronomers do. You will have to learn how to mosaic frames using the Spitzer tools developed for professional astronomers by the Spitzer Science Center. This needs a lot of disk space, and, well, a little bit of courage! And access to IDL would help a lot. | This tells you how to start from the same place professional astronomers do. You will have to learn how to mosaic frames using the Spitzer tools developed for professional astronomers by the Spitzer Science Center. This needs a lot of disk space, and, well, a little bit of courage! And access to IDL would help a lot. | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
Latest revision as of 16:12, 10 May 2012
Contents
- 1 Don't forget to try and answer the "Questions to think about ..." at the bottom of this page!
- 2 The basics: Introduction and Terminology
- 3 Downloading Data: Using the SHA, short versions
- 4 Downloading Data: Using the SHA, long version
- 5 Downloading Data: Using the SHA, a concrete example, very long version
- 6 Downloading Data: How can I find already-reduced Spitzer data?
- 7 Downloading Data: Using the SHA-searching for a list of objects
- 8 Downloading Data: How can I quickly get a mosaic of my object?
- 9 Questions to think about and things to try with the SHA
- 10 I'm ready to advance to a highly technical and in-depth discussion on processing Spitzer data.
Don't forget to try and answer the "Questions to think about ..." at the bottom of this page!
The basics: Introduction and Terminology
Introduction
The Spitzer Heritage Archive (SHA) is the permanent home for all of the data collected during the Spitzer mission, plus all the documentation you need to understand it all. The SHA is formally part of IRSA's archive holdings (no longer 'owned' by the SSC). The SHA provides a web-based interface to the Spitzer archive, and it lives here: http://irsa.ipac.caltech.edu/applications/Spitzer/SHA/ Because it is web-based, you do not need to download and install software that is platform-dependent. It should "just work" in whatever browser you use (though, for really new or really old browsers, your mileage may vary)! There is online help for the SHA -- see the help menu in the upper right (of the red menu bar). There are also several other ways to get help; see here -- look under "Spitzer Heritage Archive Documentation". The cookbook's first few chapters has detailed step-by-step recipes (one of which was originally developed for a NITARP team), the User's Guide is a standalone PDF manual, and some instructional videos are linked in as both YouTube and Flash copies.
The software that used to be the primary mechanism for pulling data from the archive is called Leopard. There might be some lingering references to Leopard on the wiki, though we have tried to clean them all out.
Basic Terminology
An individual Spitzer observation sequence is an AOR, or Astronomical Observation Request. In certain cases (often calibration or sometimes science observations), you may also see an IER, or Instrument Engineering Request. Either one involves many individual frames, as well as observer name, date of observation, object or area of the sky observed, and instrument used (IRAC, MIPS, or IRS)-- these are all part of the AOR. All of Spitzer's operations (planning, scheduling, processing) have been centered around these units (AORs or IERs). Now, for the SHA, we are starting to move away from that, but there are some things that are still only available on an AOR basis, so we really can't escape them.
The rest of the new terminology has its origin in other similar terms used in other archives. I know, I know, hard to think about astronomers using the same terms to mean the same thing across multiple telescopes and wavelengths! But we're trying...
Raw data that are fundamentally unprocessed are "Level 0" data. As NITARP folks, you should never encounter (or want to encounter, really) Level 0 data.
The individual data frames that emerge, calibrated, from the Spitzer pipeline are "Level 1," or "Basic Calibrated Data," or "BCDs." You can get just the BCDs from a region that you want; you don't have to download the whole AOR if it covers a much larger region than you want. As NITARP folks, you probably don't need these Level 1 data. But you might.
The products that come from combining these individual data frames (such as mosaics) are "Level 2," or "post-BCD," or "PBCD data." These still exist fundamentally on an AOR level, e.g., you can't get a Level 2 mosaic that is just a portion of an AOR. As NITARP folks, you probably want these data.
You can also get some higher-level processed products (which you might call "Level 3" data, but which in this context are called "Enhanced Products") through this interface. These products are supplemental data that are produced either by the SSC or donated to us by professional astronomers, and represent additional processing. For example, you can get a mosaic combining data from 7 AORs into one big mosaic, with customized (as opposed to hands-off pipeline) processing of image artifacts. Most of the enhanced products in the SHA are delivered by Legacy teams, or developed by the SSC itself. See below for more on this.
All of the images come in FITS format. (Wondering what is FITS format?) (If you are really savvy, you might also care that they are mostly single-plane FITS files. Some enhanced products will be/are multi-plane FITS.) The other format for some data is IPAC table files (.tbl extension). IPAC table format is really just plain text, with a special header. Once you get a file like this, just about anything (including Excel) can read it. (YouTube video on tbl files, how to access them, and how to get them into Excel (10min).)
Downloading Data: Using the SHA, short versions
- Option 1 : SHA Quick Start for NGC 4051 or How can I quickly get a mosaic of my object? Both are quick, text only quick start guides.
- Option 2 : Please see the first recipe in the Spitzer Data Analysis Cookbook (direct link ought to work!). This has screen snapshots. (Developed for professional astronomers. Hopefully it makes sense to you too. Let Luisa know if it doesn't.)
- Option 3 : Or, see YouTube QuickStart video (7.5 min). (Developed for professional astronomers. Hopefully it makes sense to you too. Let Luisa know if it doesn't.)
Downloading Data: Using the SHA, long version
In more detail!
- Wiki page: Using the SHA, long version
- Also see YouTube video, the long version (13.5 min).
(The wiki page was developed for you; the video was developed for professional astronomers.)
Downloading Data: Using the SHA, a concrete example, very long version
Originally developed especially for the 2010 CG4 team, but then turned into a formal chapter for the professional astronomer's Data Reduction Cookbook. This demo covers the following tasks: Use the SHA to search the Spitzer Archive for all possible and relevant CG4 observations. Use the SHA to assess which of several different observations of the same object will most quickly yield an image that you want. Select data for download, and do it.
Recipe 2 from the Cookbook (direct link should hopefully work!!)
Downloading Data: How can I find already-reduced Spitzer data?
The SHA also includes polished mosaics and source lists, with more to come!
Downloading Data: Using the SHA-searching for a list of objects
How to search for a large list of objects efficiently.
Downloading Data: How can I quickly get a mosaic of my object?
Get me a mosaic, quick! Don't bother me with preambles or complete explanations, I just want a picture. (Also see What is a mosaic and why should I care?)
Questions to think about and things to try with the SHA
Pick an object to search on, anything you want.
- How many observations are available? Which are imaging? At which bands?
- Can you find any already-processed Spitzer data on this object?
I'm ready to advance to a highly technical and in-depth discussion on processing Spitzer data.
This tells you how to start from the same place professional astronomers do. You will have to learn how to mosaic frames using the Spitzer tools developed for professional astronomers by the Spitzer Science Center. This needs a lot of disk space, and, well, a little bit of courage! And access to IDL would help a lot.
