Notes for developers

The FITS standard

The FITS format is described in the FITS Standard Document.

The CFITSIO sub-module

EasyFITS makes use of the Clang Julia package to automatically build file deps/deps.jl with constants, types, and low level functions to call the functions of the CFITSIO library with arguments of the correct type. All these are available in the EasyFITS.CFITSIO sub-module.

Calls to functions in the CFITSIO library

When calling functions of the CFITSIO library, there are several things to take care of:

  • Passing correct arguments. This is partially ensured by the type assertions in the @ccall macro. It is also necessary to check whether a pointer to some opaque structure in the library is valid.

  • Preserving objects from being destroyed while being in use. Of course, this is automatically done by Julia for Julia objects, but must be handled for references or pointers to objects provided by the CFITSIO library.

When an object obj is specified for an argument of type Ptr{T} to be passed to a C function, Julia ccall does something like:

ref = Base.cconvert(Ptr{T}, obj)
ptr = Base.unsafe_convert(Ptr{T}, ref)
result = GC.@preserve ref call_some_c_function(..., ptr, ...)

Here ref is an object (by default, Base.cconvert(Ptr{T},obj) yields obj itself) to be used with Base.unsafe_convert(Ptr{T},ref) to get the pointer and to be preserved from being garbage collected in order to warrant that the pointer remains valid.

Thanks to this mechanism, it is quite simple to ensure that valid pointers to opaque structures of the CFITSIO library be passed to a function of this library. For example, the code below is how is handled a pointer to a fitsfile C structure in our code:

isnull(ptr::Ptr{T}) where {T} = ptr === Ptr{T}(0)
check(ptr::Ptr) = isnull(ptr) ? error("invalid NULL pointer") : ptr
get_handle(file::FitsFile) = getfield(obj, :handle)
get_file(hdu::AbstractHDU) = getfield(hdu, :file)
Base.unsafe_convert(Ptr{CFITSIO.fitsfile}, obj::FitsFile) = check(get_handle(obj))
Base.cconvert(Ptr{CFITSIO.fitsfile}, hdu::AbstractHDU) = get_file(hdu)

Private methods isnull and check are introduced for readability. Private methods get_handle and get_file are two of the private accessors introduced to hide the fields of a FITS file object and let some other public properties be implemented. The former yields the pointer to the fitsfile C structure that is managed by a FitsFile object, while the latter yields the FitsFile object storing the HDU as it is the one that must be used and preserved when calling a C function requiring a pointer to a fitsfile C structure.

Note that FitsFile objects have a finalizer that automatically releases resources such as the associated fitsfile C structure when the object is garbage collected.

Helper functions

The following non-exported functions are provided for meta-programming and for dealing with the types of arguments in calls to the functions of the CFITSIO library.

EasyFITS.cfuncFunction
EasyFITS.cfunc(pfx::Union{AbstractString,Symbol}, T::Type) -> sym

yields the symbolic name of the function in the CFITSIO library whose prefix is pfx and whose suffix is deduced from the type T. Long/short function names are supported and automatically detected depending on whether pfx ends with an underscore character.

source
EasyFITS.ctypeFunction
EasyFITS.ctype(T) -> T′

yields the C type equivalent to T in CFITSIO library. This is mostly for Booleans which are treated as C char in the library, the other basic numerical types being unchanged. An error is thrown if the returned type has a different size than that of T. This is to make sure that Julia arrays with elements of type T can safely be used to store values of type T′.

Warning

This only applies to element type of arrays. For boolean scalars, a Cint is the correct type.

source
EasyFITS.cpointerFunction
EasyFITS.cpointer(arr::AbstractArray) -> ptr::Ptr{ctype(eltype(arr))}

yields a pointer to the elements of array arr that can be used in calls to functions of the CFITSIO library. Compared to Ptr{Cvoid}(pointer(arr)), this function yields a typed pointer which prevents using arguments of the wrong type.

Warning

Do not forget to use GC.@protect arr ... to avoid arr being garbage collected while its address is in use.

source

Pixel types

The following table lists conventions used by CFITSIO for pixel types, that is the BITPIX keyword in FITS image extensions.

Type CodeJulia TypeBITPIX
BYTE_IMGUInt88
SBYTE_IMGInt8
SHORT_IMGInt1616
USHORT_IMGUInt16
LONG_IMGInt3232
ULONG_IMGUInt32
LONGLONG_IMGInt6464
ULONGLONG_IMGUInt64
FLOAT_IMGFloat32-32
DOUBLE_IMGFloat64-64

Types without a value in the BITPIX column are converted by the CFITSIO library into the other signed/unsigned type using special values of the BSCALE and BZERO keywords to allow for the reciprocal conversion. This is explicitly allowed by the FITS Standard (version 4.0).

The above equivalence rules are implemented by the following two non-exported functions.

EasyFITS.type_to_bitpixFunction
EasyFITS.type_to_bitpix(T)

yields the code for FITS image pixels of type T. Argument can also be an array instance or type.

Basic numeric types are recognized by this method which may be extended by other packages to yield the CFITSIO codes equivalent to their own types. The CFITSIO constants (to be prefixed by EasyFITS.CFITSIO.) and their corresponding Julia types and standard BITPIX code are:

CFITSIO ConstantJulia TypeBITPIX
BYTE_IMGUInt88
SBYTE_IMGInt8
SHORT_IMGInt1616
USHORT_IMGUInt16
LONG_IMGInt3232
ULONG_IMGUInt32
LONGLONG_IMGInt6464
ULONGLONG_IMGUInt64
FLOAT_IMGFloat32-32
DOUBLE_IMGFloat64-64

Note that CFITSIO can read/write non-standard pixel types (those without a BITPIX value above) by setting keywords BSCALE and BZERO with special values as explicitely allowed by the FITS Standard (version 4).

source

Array data types

The following table lists conventions used by CFITSIO for array element types.

Type CodeC TypeShort SuffixLong Suffix
TLOGICALcharl_log
TBYTEunsigned charb_byt
TSBYTEsigned charsb_sbyt
TUSHORTunsigned shortui_usht
TSHORTshorti_sht
TUINTunsigned intuk_uint
TINTintk_int
TULONGunsigned longuj_ulng
TLONGlongj_lng
TULONGLONGunsigned long longujj_ulnglng
TLONGLONGlong longjj_lnglng
TFLOATfloate_flt
TDOUBLEdoubled_dbl
TCOMPLEXfloat complexc_cmp
TDBLCOMPLEXdouble complexm_dblcmp
TSTRINGchar*s_str
TBITx_bit
u_null

Complex types float complex and double complex are stored as pairs of single/double precision floating-point values (this is not guaranteed by C99 standard so strict equivalence does not hold here).

The above equivalence rules are implemented by the following two non-exported functions.

EasyFITS.type_to_codeFunction
EasyFITS.type_to_code(T)

yields the CFITSIO type code for a keyword value or table cells of type T. Argument can also be an array instance or type.

Basic numeric types and string types are recognized by this method which may be extended by other packages to yield the CFITSIO codes equivalent to their own types. The CFITSIO type constants (to be prefixed by EasyFITS.CFITSIO.) and their corresponding C and Julia types are:

CFITSIO ConstantC TypesJulia Types
TLOGICALcharCchar, Bool
TBYTEunsigned charUInt8, Bool
TSBYTEsigned charInt8
TUSHORTunsigned shortCushort
TSHORTshortCshort
TUINTunsigned intCuint
TINTintCint
TULONGunsigned longCulong
TLONGlongClong
TULONGLONGunsigned long longCulonglong
TLONGLONGlong longClonglong
TFLOATfloatCfloat
TDOUBLEdoubleCdouble
TCOMPLEXfloat complexComplex{Cfloat}
TDBLCOMPLEXdouble complexComplex{Cdouble}
TSTRINGchar*
TBIT
source

Column data types

The following table lists the correspondences between the TFORMn letter in FITS table extensions and the column data type.

Type CodeJulia TypeTFORMDescription
TLOGICALBool’L’Logical (1 byte)
TBIT’X’Bit (special)
TBYTEUInt8’B’8-bit unsigned integer
TSHORTInt16’I’16-bit signed integer
TLONGInt32’J’32-bit signed integer
TLONGLONGInt64’K’64-bit signed integer
TSTRINGString’A’Character (1 byte, used for strings)
TFLOATFloat32’E’32-bit floating point
TDOUBLEFloat64’D’64-bit floating point
TCOMPLEXComplexF32’C’64-bit complex
TDBLCOMPLEXComplexF64’M’128-bit complex
’P’32-bit array descriptor
’Q’64-bit array descriptor

A few non-standard TFORM letters are allowed by CFITSIO. These are converted by the library into other types using TSCALE and TZERO keywords to allow for the reciprocal conversion following the same principles as for the BITPIX code and the BSCALE and BZERO keywords.

Type CodeJulia TypeTFORMDescription
TSBYTEInt8’S’8-bit signed integer
TUSHORTUInt16’U’16-bit unsigned integer
TULONGUInt32’V’32-bit unsigned integer
TULONGLONGUInt64’W’64-bit unsigned integer

The Type Code column indicates the code used by CFITSIO (it is not always consistent with the C types as defined in the above table, so my guess is that this code is only used to keep track of the column data type internally).

The above equivalence rules are implemented by the following two non-exported functions.

EasyFITS.type_to_letterFunction
EasyFITS.type_to_letter(T)

yields the letter of the TFORMn keyword representing table cells of type T in FITS. Argument can also be an array instance or type.

source
EasyFITS.type_from_letterFunction
EasyFITS.type_from_letter(c) -> T

yields the Julia type T corresponding to CFITSIO column type letter c as assumed for the TFORMn keywords.

source