Sloan Digital Sky Survey
Review of Data Processing and Distribution Operations
Technical Summary of the Photometric Pipeline
Jill Knapp and Robert Lupton
Princeton University
June 20, 2000
Dear Reader:
We are beginning, or perhaps tiptoeing around, writing up the technical
paper describing the Photometric Pipeline. The following outline
is
a summary of what Photo does:
PHOTO paper: Outline
====================
Draft of June 20 2000
The first two chapters are background and expected to get into few details.
1. Introduction
===============
The SDSS
Filters, CCD mosaic, data taking
CCD photometry: particular problems and characteristics
TDI mode
pixel scale
strips and stripes
lateral overlaps
expected PSF
time to cross array
expected/observed sky brightness
expected PS sensitivities
expected surface brightness sensitivities
saturation
position calibration: Photo <-> Astrom
photometric calibration: Photo <-> MTpipe
What Photo has to do
Data rate
Execution time
Goals have been met
Outline of the rest of the paper
Plans for distribution of code
useful URLs
Tables: Filters, nominal wavelengths, widths, PS sensitivities
Sky coverage, size of a frame, pixel size, lateral
overlap area,
time to cross array, integration time
Figures: pretty pic of camera, or diagram of filter layout
pretty color pic of a piece of the sky
2. Photo: Overall Architecture
==============================
DA system and astroline
Motivation for architecture
Frames
needs:
PSP
needs:
SSC
(I have put it in this order because we need to explain why
we need
PSP and in turn why we need SSC)
Necessary inputs to Frames
Data file formats and nomenclature
Table: list of inputs:
postage stamps
darks
bias
flats
fang & gang files
astroline stamps
astrom stars
astrom and ptpipe trans structures and files
Figures: data flow through photometric pipelines
3. SSC
======
Inputs from astroline
cutting frames and overlaps
aligning frames to make a field
("data startup cost" goes here?)
cutting postage stamps
definition of different kinds of postage stamp
outputs
Table: inputs to SSC
outputs from SSC
(do we need this?)
4. PSP-1: Overview and Functionality
====================================
Tasks of PSP
Inputs to PSP
Outputs from PSP
Analysis of idB files: smoothing/averaging lengths
overscan
flat field
sky
Postage stamps: different types and their purpose
PSF
wings
frame stars
ghosts
Wings of bright stars
diffraction spikes
ghosts (?)
Tables: Inputs
Outputs
Figures: data flow through PSP
montage of postage stamps
variation of sky & flat field on a typical night
5. PSP-2: PSF
======================
**Note: this will probably be a separate paper**
The problem
Optics
Seeing
Photometry
Modeling the PSF
Optics, seeing and spatial variation
stationarity
basis functions
KL decomposition
required number of stamps per frame
Stokes parameters
rejection
using modelled PSF
aperture corrections
PSF widths
Interpolation of PSF over a run
Dealing with too few postage stamps
Science: some remarks about seeing
atmospheric modeling
promise further analysis
Table: PSF decomposition: terms and coefficients
Figures: Aperture corrections vs time: "before" and "after"
Model vs real PSF across frame, and residuals
Variation of PSF width across camera
Variation of PSF width as a function of time
QU plots
width plots
6. Frames-1 Introduction
=========================
Inputs and outputs (general)
A walk through Frames
what gets done where
what gets done when
data problems
pixel size and band limit
bad columns
CRs
bleed trails
wings, ghosts
variable PSFs
Image analysis
Tables: Inputs
Outputs (summary)
Figures: a frame of raw data
the same frame corrected
data flow through Frames
7. Frames-2: Correction of data defects
=======================================
Introduction
description of a frame
saturation levels in each filter
application of bias
application of flat
Bad columns
typical number of bad columns
origin of bad column map
correcting for bad column
Cosmic rays
finding cosmics
correcting for cosmics
number of cosmics in a typical frame
Bleed trails
Serial register artifacts
Outputs: corrected frames
Figures: performance of interpolation across columns
"before" and "after" of
cosmic ray
bad column
bleed trail
7. Frames-3: Finding and Combining Objects
==========================================
First pass: find bright objects
saturated stars (already known)
subtract bright objects
wings of saturated stars
(?exploiting wings to measure magnitude?)
Second pass: finding objects to the limit
psf filtering
find objects
binning/smoothing
finding more objects
blank sky objects
Growing objects
Combining filters
Blank sky objects
Definition of atlas images
Outputs
atlas images
(compression of atlas images)
object subtracted binned frames
masks
Tables: Limits
(faintest & brightest PSs, lowest and
highest
surface brightness)
How much sky is lost
Outputs
Figures: `before' and 'after' subtraction of saturated star with
wings
Frame with atlas image boundaries and centroids
of all objects
Example atlas images
Frame reconstructed from Atlas Images
8. Frames-4: Measure Objects
============================
(understood: everything below is followed by "& uncertainties")
sky subtraction: global vs local
sub-pixellization
centroiding, sinc-shifting and astrometric offsets between bands
sector array and radial profiles
objects at frame edges
processing flags
photometry
psf
aperture
3" aperture
radial profile
isophotal
luptitudes and procedure for "upper limits"
Petrosian magnitudes
Petrosian radii
shapes
ellipticities
position angles
Stokes parameters
isophotal shape
model fits
definition
library
psf convolution
likelihoods
model magnitudes and model parameters
model colors
star/galaxy separation
**this may be a separate paper**
roughness parameter(s)
PSF fraction
Figures: map of sector array
PSF vs model vs aperture magnitudes for point sources
deV/exponential/psf/likelihood prism diagrams vs magnitude
sample color color and color magnitude diagrams
Tables: Photo outputs
Photo flags
9. Frames-5: Astrometric and Photometric Calibration
====================================================
This should be short. There should be a reference to whatever
photometric
calibration paper we have/will have.
Astrometric calibration:
Anomalous refraction
Map of CCD array and position information
astrometric trans structures (w reference to ASTROM
paper)
centroiding and transfer objects
Photometric calibration:
photometric trans structures
calibration stars
final calibration
Figures: sample color-color and color-magnitude diagrams
color pictures of some galaxies
star and galaxy counts
10. Frames-6: Deblending
========================
**note: this will be a separate paper, already drafter**
Statement of the problem
Definition of a blended object
decomposition algorithm
problem objects
Tables: number of blended objects as a function of magnitude
Figures: images of several examples of parents and children
schematic to show how the algorithm works
11. Frames-7: Moving Objects
=============================
The problem
expected proper motions
very fast objects
moving objects as contaminants
finding moving objects
flags
prospects for future (Kuiper belt etc)
Figures: velocities showing asteroids
(color-magnitude plots of asteroids)
(color-color plots of asteroids)
12. Photo Summary: Outputs and Performance
==========================================
Outputs
Processing flags
performance on several platforms
profiles: memory, timing
Tables: Measured parameters
Output parameters
Flags
Output formats
Figures: timing/memory diagrams
13. Testing and QA
==================
History
tracking problems
testing PSP: simulations
simulations: description
HST fields, including the Groth strip
clusters
overlaps: lateral, half field, repeat
The QA test suite: tracking Photo's performance
Tables: known regions observed
Figures: simulated images
scan of globular cluster
scan of HST field
14. Summary
===========
Photo performance: accuracy, completeness, speed
Information produced
list of web sites for more information
15. Future
==========
Coaddition
cosubtraction
atlas image analysis
SDSS boilerplate etc
plans for releasing the code
|