Make your HiPS in 10 steps

By P.Fernique (last update July 2022 - additions in green)

... or how to build a Hierarchical Progressive Survey from your image (or cube) collection.

  1. The observations do not fully cover the whole image arrays (ex: HST, Chandra):
    • If your image contains not exposed pixels, check if the BLANK FITS keyword is correctly set to the value of these pixels. If it is not the case, specify it via the parameter blank=xx (generally blank=0). Notice that the BLANK value is expressed as the FITS pixel coding (BSCALE and BZERO not applied), and not in physical quantity.
    • Check if there are bad pixels at the borders of the image. You can remove them via the parameter border=nn.
    • If the observation is circular or rectangular (ex: GALEX), you can specify it via the parameter shape=sss
    • If the observation is complex and can be discribed by a polygon (ex: Schmidt plate), you can specify a polygon, either associated to a specifical image (same name, .fov extension), or for a collection of images set together in a sub-directory - the file describing the polygon must have the same name of the directory, .fov extension. You must use fov=true parameter to take into account these polygon files.
      Tips:Use Aladin Desktop with the tool Tag for generating the list of coordinates. Cut-&-paste the result in a clip board for keeping only the RA and DEC columns (one per line).
  2. The sky back grounds of the observations are not homogeneous:
    • If the sky back ground value is specified in the FITS header, you can indicate the corresponding FITS key to Aladin/Hipsgen via the parameter skyval=XXXX (for instance skyval=BACKGRND)
    • Otherwise, you can try with skyval=true for asking Aladin/Hipsgen to compute an estimation of each sky back ground.
  3. The original images are packaged in Multi-Extension FITS:
    Specify the good extension via the parameter hdu=nn, or all hdu=all.
  4. You would like to modify the BITPIX, notably if the original images are coded in a surprising bit range (ex: -64 for counting a few photons...) and/or you would like to minimize the HiPS size:
    • Choose a good target BITPIX (ex: 16 means 2^16 = 65536 grey levels typically enough for most of detectors)
    • Specify this target BITPIX via the parameter bitpix=nn.
    • Determine manually the physical range of original valid pixels and specify them thanks to pixelRange="min max" parameter. Otherwise Aladin/Hipsgen will do an automatical estimation on the first image of the collection, or on a specifical image specified by img="path" parameter. The min and max values are expressed in physical quantities (BSCALE, BZERO applied).
  5. Your image collection are in JPEG or PNG, and not in FITS:
    • For each image, you must provided the associated astrometrical calibration - AVM tags or WCS solution - either directly inside the JPEG (resp. PNG) comment segment, or for WCS solution, in a separated file same name, .hhh extension) or . The WCS solution can be expressed as a FITS header (80 character alignement), or with basic ASCII line separators.
    • Specify the resulting tile format via the parameter color=jpeg or color=png (or color=true to keep the same format that the original images).
    Tips: If your JPEG or PNG images are quite huge (>100 Megapixels), do not hesitate to split your images in sub-images of 1024x1024 pixels. The process will be really faster. Do not forget to adjust the WCS solutions according to.
  6. Your image is a HEALPix fits map:
    In this case, your original data will be not resampled, but just mapped into HiPS tiles. You have to specify the filename of the HEALPix map instead of the input directory:
    java -Xmx2000m -jar Aladin.jar in="YourHealpixMap.fits
  7. You would like to restart partially the process:
    • You can specify on which region you want to restart the process. Use the parameter moc=sss where sss is a inline ASCII MOC or a path to a MOC file.
      Have a look on mode=xxx parameter for controlling the pixels and tiles merging/replacing behaviour...
    • You can also launch Aladin/Hipsgen step by step by specifying explicitely each action. The default action list is INDEX TILES PNG CHECKCODE DETAILS UPDATECHECKCODE. You can restart one specifical action, for instance relaunch PNG tile computation
      java -jar AladinBeta.jar -hipsgen in=Data PNG
  8. You would like to generate a colored HiPS combining several survey filters (2 or 3):
    • For each filter, you have to create before the grey level HiPS.
      → we assume "redHips", "greenHips" and "blueHips" paths.
    • After that, launch Aladin/Hipsgen RGB action with these parameters:
      java -jar AladinBeta.jar -hipsgen inRed=redHips inGreen=greenHips inBlue=blueHips RGB
      You can adjust the color mapping scale for each color component thanks to cmRed, cmBlue and cmBlue parameters (see details below)
    • From Aladin version 11.9 onwards, it is possible to view the expected HiPS colour result before calculation and to retrieve the parameters for the corresponding Hipsgen command.
  9. You would like to generate your HiPS incrementally
    This facility is useful for incorporating new observations in an already existing HiPS without restarting all the process (or if you have no longer access to the original images). This option is only supported in FITS mode.
    • Your original HiPS had to be generated with the -live parameter
      java -jar AladinBeta.jar -hipsgen -live in=... out=DataHiPS ...
    • Append new observations thanks to APPEND action:
      java -jar AladinBeta.jar -hipsgen in=... out=DataHiPS APPEND
    • Note that incremental addition will require twice the disk space (storage of weight tiles). Use only if necessary. It is also possible to delete the weight tiles after the process is finished (=> action CLEANWEIGHT)
  10. You would like to concatenate two HiPS
    This functionality is possible between two compatible HiPS (same coordinate system, same order, same format, same settings for FITS tiles). The best results will be obtained in FITS mode with the -live option.
    • Generate the first HiPS:
      java -jar AladinBeta.jar -hipsgen in=... out=Hips1 ...
    • Generate the second HiPS:
      java -jar AladinBeta.jar -hipsgen in=... out=Hips2 ...
    • Concatenate the second HiPS into the first one thanks to CONCAT action:
      java -jar AladinBeta.jar -hipsgen in=Hips2 out=Hips1 CONCAT
  11. You would like to patch (or regenerate) a HiPS on a specific region
    You can restrict the HiPS target size thanks to the region=moc parameter. This "MOC" can be described via an inline string, or via a real MOC file.
    Associated to the appropriated mode parameter, notably the value OVERWRITE (pixel replacement), or KEEP (pixel completion), you will have the full control to decide how to patch an already existing HiPS.
    java -jar AladinBeta.jar -hipsgen in=Data region="3/131,132 4/565" mode=OVERWRITE ...
    Note: For advanced mode parameters (REPLACETILE, KEEPTILE) take care that you need to have access to all the original images ("in" directory must be unchanged). If you want to add new observations see the previous item)
  12. You would like to generate the progenitors links:
    This facility generates links for providing an access to the original images.
    • Launch Aladin/Hipsgen DETAILS action if required (already done by default)
    • Edit the file DataHiPS/HpxFinder/metadata.xml and follow the inside explanations, notably for re-incorporating the metadata memorized thanks to fitskey="xxx" parameter.
  13. You want to clone a remote HiPS:
    This feature allows you to mirror a HiPS. Be sure to check the copy and redistribution rights beforehand.
    Launch Aladin/Hipsgen with MIRROR action:
    java -jar AladinBeta.jar -hipsgen in=http://remote/url MIRROR
    Note: The resulting HiPS will be stored in the current directory, and the metadata will be automatically adapted for describing a "mirrored" HiPS.

  1. If the images are relatively small (< 3 Megapixels) and the collection does not have a lot of overlays (<6 images on the same location), set the parameter: partitioning=false.
  2. If the images have already homogeneous sky back ground, set the parameter: mixing=false.
  3. If your have to generate JPEG or PNG tiles:
    • JPEG is 4 to 20 times faster than PNG (but does not manage transparency and introduces artefacts).
    • PNG encoder time can vary by a factor 10 depending of the sky background noise. More you remove the sky background noise by adjusting the min pixel cut, faster the PNG encoder will work.
  4. For checking the result before the end it is possible to have a quick look of the current result:
    1. Stop Aladin/Hipsgen: kill -STOP pid, or CTRL+Z
    2. Launch this command: java -jar AladinBeta.jar -hipsgen in=Data ALLSKY
    3. Look the current result: java -jar AladinBeta.jar DataHiPS
    4. Restart the process: kill -CONT pid, or fg

Usage: java -jar HipsGen in=file|dir [otherParams ... ACTIONs ...]
       java -jar HipsGen -param=configfile

The config file must contain these following options, or use them
directly on the comand line :

Available actions (by default: "INDEX TILES PNG CHECKCODE DETAILS"):
   INDEX      Build spatial index (in HpxFinder directory) + MOC index
   TILES      Build all true value pixel tiles (FITS) + Allsky.fits + MOC
   JPEG       Build all preview tiles (JPEG) + Allsky.jpg
   PNG        Build all preview tiles (PNG) + Allsky.png
   CHECKCODE  Compute+store the check codes (and the size) associated to the target HiPS
   RGB        Build and RGB HiPS based on 2 or 3 other HiPS
   MOC        (Re)build the MOC (MultiOrder Coverage map)
   ALLSKY     (Re)build all Allsky files + index.html
   TREE       (Re)build HiPS tree structure from already existing tiles
   MAPTILES   Build all FITS tiles from a HEALPix Fits map
   APPEND     Append new images/cubes to an already existing HiPS
   CONCAT     Concatenate one HiPS to another HiPS
   CUBE       Create a HiPS cube based on a list of HiPS (; separated)
   CLEANFITS  Delete all FITS tiles and Allsky.fits
   DETAILS    Adapt HiPS index for supporting the "detail table" facility
   MAP        Build an HEALPix map from the HiPS tiles
   MIRROR     Mirror a remote HiPS locally
   UPDATE     Upgrade HiPS metadata additionnal files to last HiPS standard
   CHECK      Verify the HiPS based on the check codes previously computed (CHECKCODE)
   CHECKDATASUM  Verify the FITS tiles based on internal FITS DATASUM and the check code
   LINT       Check HiPS IVOA 1.0 standard compatibility

Required parameter:
   in=dir                  Source image directory (FITS or JPEG|PNG +hhh or HiPS),
                           unique image or HEALPix map file
   creator_did=id      HiPS identifier (syntax: AUTHORITY/internalID)

Basic optional parameters:
   out=dir             HiPS target directory (default ./+"AUTHORITY_internalID")
   hips_creator=name   Name of the person|institute who builds the HiPS
   hdu=n1,n2-n3,...|all List of HDU numbers (0 is the primary HDU - default is 0)
   blank=nn|key        Specifical BLANK value, or alternate BLANK fits keyword
   color=jpeg|png      The source images are colored images (jpg or png) and the tiles
                       will be produced in jpeg (resp. png)
   -n                  Just print process information, but do not execute it.
   -f                  clear previous computations

Advanced optional parameters:
   hips_order=nn       Specifical HEALPix order - by default, adapted to the original
   hips_pixel_bitpix=nn Specifical target bitpix (-64|-32|8|16|32|64)
   hips_pixel_cut=min max Specifical pixel cut and/or transfert function for PNG/JPEG 8 bits
                       conversion - ex: "120 140 log")
   hips_data_range=min max Specifical pixel value range (required for bitpix
                       conversion, or for removing bad pixels - ex: "-5 110")
   img=file            Specifical reference image for default initializations 
   mode=xx             Coadd mode, action dependent:
                       .TILES (restart) -> pixel impact: *OVERWRITE*|KEEP|SUM|AVERAGE
                                  tile impact: REPLACETILE|KEEPTILE
                       .CUBE   -> tile copy: LINK|*COPY*
                       .CONCAT -> pixel impact: OVERWRITE|KEEP|ADD|DIV|MUL|*AVERAGE*
                                  tile impact : REPLACETILE|KEEPTILE
   partitioning=true|false|nnn True for cutting large original images in blocks of nnn x nnn 
                       (default is true, nnn=512 )
   region=moc          Specifical MOC region to compute (ex: 3/34-38 50 53)
                       or Moc.fits file (all sky by default)
   fov=true|x1,y1..    Observed regions by files.fov or global polygon (in FITS convention).
   hips_min_order=nn   Specifical HEALPix min order (only for DETAILS action)
   hips_frame          Target coordinate frame (equatorial|galactic)
   hips_tile_width=nn  Specifical tile width (pow of 2) - default 512
   hips_status=xx      HiPS status (private|public clonable|clonableOnce|unclonable)
                       (default: public clonableOnce)
   cache=dir           Directory name for an alternative cache disk location
   cacheSize=nn        Alternative cache disk size limit (in MB - default 1024
   cacheRemoveOnExit=true|false Remove or not the cache disk at the end - default true
   maxThread=nn        Max number of computing threads
   target=ra +dec      Default HiPS target (ICRS deg)
   targetRadius=rad    Default HiPS radius view (deg)
   pilot=nnn           Pilot test limited to the nnn first original images.
   verbose=n           Debug information from -1 (nothing) to 4 (a lot)
   -live               incremental HiPS (keep weight associated to each HiPS pixel)
   -notouch            Do not touch the hips_release_date
   -color              Colorized console log messages

Specific optional parameters:
.INDEX action:
   fitskeys=list       Fits key list (blank separator) designing metadata FITS keyword value 
                       to memorized in the HiPS index
.TILES action:
   shape=...           Shape of the observations (ellipse|rectangle)
   border=...          Margins (in pixels) to ignore in the original observations
                       (top left bottom right or constant)
   skyval=key|auto|%info|%min %max   Fits key to use for removing a sky background, or auto
                       detection or percents of pixel histogram kept (central ex 99, or
                       min max ex 0.3 99.7)
   exptime=key         Fits key to use for adjusting variation of exposition
   fading=true|false   False to avoid fading effect on overlapping original images 
                       (default is false)
   mixing=true|false   False to avoid mixing effect on overlapping original
                       images (default is true [pixel average])
   pixelGood=min [max] Range of pixel values kept
   maxRatio=nn         Max height/width pixel ratio tolerated for original obs 
                       (default 2, 0 for removing the test)
   method=m            Method (MEDIAN|MEAN|FIRST) (default MEDIAN) for aggregating colored 
                       compressed tiles (JPEG|PNG)
.RGB action:
   inRed               HiPS red path component, possibly suffixed by cube index (ex: [1])
   inGreen             HiPS green path component, possibly suffixed by cube index (ex: [1])
   inBlue              HiPS blue path component, possibly suffixed by cube index (ex: [1])
   cmRed               Colormap parameters for HiPS red component (min [mid] max [fct])
   cmGreen             Colormap parameters for HiPS green component (min [mid] max [fct])
   cmBlue              Colormap parameters for HiPS blue component (min [mid] max [fct])
   luptonQ=x           Q coef Lupton RGB builder (default auto)
   luptonS=x/x/x       scale coefs Lupton RGB builder (default auto)
   luptonM=x/x/x       m coefs Lupton RGB builder (default auto)
   filter=gauss        Gaussian filter applied on the 3 input HiPS

.MOC action:
   mocOrder=s          Specifical MOC order.
                       s-spaceOrder, t-timeOrder, maxLimit, degradation rule
.MAP action:
   nside=nn            HEALPix map NSIDE - by default 2048

.MIRROR action:
   tileTypes           List of tile format to copy
   split='size;altPath]' multi disk partition split (ex: 300g;/hips/part2)
   -nocheck            Do not check date&size of local tiles
   -nice               Slow download for avoiding to overload remote http server

© Université de Strasbourg/CNRS

    • Contact