dfmux
dfmux contains code and dataclasses to collect and store data from the DfMux system, particulary focusing on the ICE system used for SPT3G and PB2. A general overview of how the modules here interact can be found in the Data Acquisition section of the manual.
API Documentation
Frame Objects
Serializable objects that may be stored as entries in G3Frame
objects.
- class spt3g.dfmux.DfMuxBoardSamples()
- class spt3g.dfmux.DfMuxBoardSamples(arg: DfMuxBoardSamples)
- class spt3g.dfmux.DfMuxBoardSamples(arg: Iterable)
Bases:
_DfMuxBoardSamplesBaseMap
,G3FrameObject
Container structure for samples from modules on one board, mapping 0-indexed module and block IDs to a dfmux.DfMuxSample.
Signature 2: Copy constructor
Signature 3: Iterable constructor
- Complete() bool
True if this structure contains data from all expected modules and blocks
- clear()
Remove all items from the mapping.
- copy() DfMuxBoardSamples
Return a shallow copy of the mapping.
- get(key: int, default: object = None) object
Return the value for key if key is in the mapping, else default.
- pop(key: int) DfMuxSample
- pop(key: int, default: object) object
Signature 1: Remove specified key and return the corresponding value. If the key is not found, raise a KeyError
Signature 2: Remove specified key and return the corresponding value. If the key is not found, return the default
- update(items: Iterable = [], **kwargs)
Update mapping from iterable/mapping.
- values() ValuesView
Return an object providing a view on the mappings’s values.
- property nblocks: int
Number of sub-module blocks expected to report from this board
- property nchannels: int
Number of channels per block expected to report from this board
- property nmodules: int
Number of modules expected to report from this board
- class spt3g.dfmux.DfMuxChannelMapping(arg: DfMuxChannelMapping)
- class spt3g.dfmux.DfMuxChannelMapping()
Bases:
G3FrameObject
Bolometer wiring information. Module and channel IDs are stored zero-indexed, but be aware that they often printed one-indexed for compatibility with pydfmux.
Signature 1: Copy constructor
- Description() str
Long-form human-readable description of the object
- Summary() str
Short (one-line) description of the object
- property board_ip: int
IP Address of the board, encoded as an int using struct
- property board_serial: int
Serial number of the readout board to which this channel is attached.
- property board_slot: int
Crate slot of the board to which this channel is attached or -1 if the board is not in a crate.
- property channel: int
0-indexed channel number on the parent module/SQUID
- property crate_serial: int
Serial number of the crate in which the readout board is housed or -1 if the board is not in a crate.
- property module: int
0-indexed module/SQUID ID of the channel
- class spt3g.dfmux.DfMuxHousekeepingMap()
- class spt3g.dfmux.DfMuxHousekeepingMap(arg: DfMuxHousekeepingMap)
- class spt3g.dfmux.DfMuxHousekeepingMap(arg: Iterable)
Bases:
_DfMuxHousekeepingMapBaseMap
,G3FrameObject
Container structure for housekeeping data from all DfMux boards, indexed by board serial number.
Signature 2: Copy constructor
Signature 3: Iterable constructor
- clear()
Remove all items from the mapping.
- copy() DfMuxHousekeepingMap
Return a shallow copy of the mapping.
- get(key: int, default: object = None) object
Return the value for key if key is in the mapping, else default.
- pop(key: int) HkBoardInfo
- pop(key: int, default: object) object
Signature 1: Remove specified key and return the corresponding value. If the key is not found, raise a KeyError
Signature 2: Remove specified key and return the corresponding value. If the key is not found, return the default
- update(items: Iterable = [], **kwargs)
Update mapping from iterable/mapping.
- values() ValuesView
Return an object providing a view on the mappings’s values.
- class spt3g.dfmux.DfMuxMetaSample()
- class spt3g.dfmux.DfMuxMetaSample(arg: DfMuxMetaSample)
- class spt3g.dfmux.DfMuxMetaSample(arg: Iterable)
Bases:
_DfMuxMetaSampleBaseMap
,G3FrameObject
Container structure for coincident samples from all boards. Individual board data, stored in dfmux.DfMuxBoardSamples classes, is contained indexed by board serial number.
Signature 2: Copy constructor
Signature 3: Iterable constructor
- clear()
Remove all items from the mapping.
- copy() DfMuxMetaSample
Return a shallow copy of the mapping.
- get(key: int, default: object = None) object
Return the value for key if key is in the mapping, else default.
- pop(key: int) DfMuxBoardSamples
- pop(key: int, default: object) object
Signature 1: Remove specified key and return the corresponding value. If the key is not found, raise a KeyError
Signature 2: Remove specified key and return the corresponding value. If the key is not found, return the default
- update(items: Iterable = [], **kwargs)
Update mapping from iterable/mapping.
- values() ValuesView
Return an object providing a view on the mappings’s values.
- class spt3g.dfmux.DfMuxSample(arg: DfMuxSample)
- class spt3g.dfmux.DfMuxSample(time: int, nsamples: int)
Bases:
IntVector
,G3FrameObject
Samples from all channels on one readout module, stored with I and Q interleaved, such that the first element is channel 1 I, followed by channel 1 Q, followed by channel 2 I, etc.
Signature 1: Copy constructor
- Description() str
Long-form human-readable description of the object
- Summary() str
Short (one-line) description of the object
- class spt3g.dfmux.DfMuxWiringMap()
- class spt3g.dfmux.DfMuxWiringMap(arg: DfMuxWiringMap)
- class spt3g.dfmux.DfMuxWiringMap(arg: Iterable)
Bases:
_DfMuxWiringMapBaseMap
,G3FrameObject
Mapping from logical detector ID string (same as used in timestreams) to wiring information (the board, module, and channel to which a given detector is connected)
Signature 2: Copy constructor
Signature 3: Iterable constructor
- clear()
Remove all items from the mapping.
- copy() DfMuxWiringMap
Return a shallow copy of the mapping.
- get(key: str, default: object = None) object
Return the value for key if key is in the mapping, else default.
- pop(key: str) DfMuxChannelMapping
- pop(key: str, default: object) object
Signature 1: Remove specified key and return the corresponding value. If the key is not found, raise a KeyError
Signature 2: Remove specified key and return the corresponding value. If the key is not found, return the default
- update(items: Iterable = [], **kwargs)
Update mapping from iterable/mapping.
- values() ValuesView
Return an object providing a view on the mappings’s values.
- class spt3g.dfmux.HkBoardInfo(arg: HkBoardInfo)
- class spt3g.dfmux.HkBoardInfo()
Bases:
G3FrameObject
Mux board status. Includes both configuration and sensor readings for board generic quantities and a list of quantities for the mezzanines.
Signature 1: Copy constructor
- Description() str
Long-form human-readable description of the object
- Summary() str
Short (one-line) description of the object
- property currents: spt3g.core._G3MapDoubleBaseMap
Dictionary of data from on-board current sensors
- property fir_stage: int
Sample rate encoded as the “FIR Stage”. Smaller numbers are faster and grow by factors of two with each decrement
- property firmware_name: str
Firmware name
- property firmware_version: str
Firmware version
- property is128x: bool
Boolean for whether 128x firmware is running
- property mezz: HkMezzanineInfoMap
1-indexed mapping from mezzanine ID to mezzanine-specific data
- property serial: str
Board serial number
- property temperatures: spt3g.core._G3MapDoubleBaseMap
Dictionary of data from on-board temperature sensors (C)
- property timestamp_port: str
Source of timestamps on board
- property voltages: spt3g.core._G3MapDoubleBaseMap
Dictionary of data from on-board voltage sensors
- class spt3g.dfmux.HkChannelInfo(arg: HkChannelInfo)
- class spt3g.dfmux.HkChannelInfo()
Bases:
G3FrameObject
Mux channel status (configuration and sensors). Usually a part of an HkModuleInfo.
Signature 1: Copy constructor
- Description() str
Long-form human-readable description of the object
- Summary() str
Short (one-line) description of the object
- property bias_frequency: float
NCO-adjusted bias frequency (mkid only)
- property carrier_amplitude: float
Carrier amplitude in normalized units (0-1).
- property carrier_frequency: float
Carrier frequency in standard frequency units
- property carrier_phase: float
Carrier phase in standard angle units (mkid only)
- property channel_number: int
1-indexed channel number.
- property dan_accumulator_enable: bool
True if DAN accumulator enabled
- property dan_feedback_enable: bool
True if DAN control of the nuller is enabled
- property dan_gain: float
DAN gain in board units
- property dan_railed: bool
True if DAN has railed.
- property dan_streaming_enable: bool
True if streamer packets are from DAN rather than demod
- property demod_frequency: float
Frequency of the demodulator in standard frequency units.
- property demod_phase: float
Demodulator phase in standard angle units (mkid only)
- property external_phase: float
Phase angle associated with delays external to the readout board (mkid only)
- property i_slope: float
Calibration coefficient in units of counts per frequency (mkid only)
- property internal_phase: float
Phase angle associated with delays internal to the readout board (mkid only)
- property loopgain: float
Measured loopgain of the detector as stored by the control software tuning script.
- property nuller_amplitude: float
Nuller amplitude in normalized units (0-1)
- property nuller_phase: float
Nuller phase in standard angle units (mkid only)
- property q_slope: float
Calibration coefficient in units of counts per frequency (mkid only)
- property rfrac_achieved: float
Achieved resistance of the detector when tuned as a fraction of rnormal, as stored by the control software tuning script.
- property rlatched: float
Resistance of the detector when latched in standard impedance units as stored by the control software tuning script.
- property rnormal: float
Resistance of the detector when normal in standard impedance units as stored by the control software tuning script.
- property state: str
String code (“latched”, “tuned” etc.) describing the state of this detector stored by the control software
- class spt3g.dfmux.HkMezzanineInfo(arg: HkMezzanineInfo)
- class spt3g.dfmux.HkMezzanineInfo()
Bases:
G3FrameObject
Mux mezzanine status
Signature 1: Copy constructor
- Description() str
Long-form human-readable description of the object
- Summary() str
Short (one-line) description of the object
- property currents: spt3g.core._G3MapDoubleBaseMap
Dictionary of measured currents on mezzanine
- property modules: HkModuleInfoMap
1-indexed list of housekeeping data from readout modules on this mezzanine
- property part_number: str
Mezzanine part ID (usually “MGMEZZ04”)
- property power: bool
True if on
- property present: bool
True if exists
- property revision: str
Mezzanine revision number
- property serial: str
Mezzanine serial number
- property squid_controller_power: bool
True if SQUID controller board powered up
- property squid_controller_temperature: float
Measured temperature of SQUID controller board (C)
- property squid_heater: float
Power level of SQUID header control
- property temperature: float
Mezzanine temperature (C)
- property voltages: spt3g.core._G3MapDoubleBaseMap
Dictionary of measured voltages on mezzanine
- class spt3g.dfmux.HkModuleInfo(arg: HkModuleInfo)
- class spt3g.dfmux.HkModuleInfo()
Bases:
G3FrameObject
Mux module status
Signature 1: Copy constructor
- Description() str
Long-form human-readable description of the object
- Summary() str
Short (one-line) description of the object
- property carrier_gain: int
Carrier gain code, in board-specific units
- property carrier_railed: bool
True if carrier has recently hit a DAC rail
- property channels: HkChannelInfoMap
Mapping from 1-indexed channel numbers to channel housekeeping data
- property demod_gain: int
Demod gain code, in board-specific units
- property demod_railed: bool
True if demod input has recently hit one of the ADC rails
- property module_number: int
1-indexed module number on this mezzanine
- property nco_frequency: float
NCO frequency in standard frequency units (mkid only)
- property nuller_gain: int
Nuller gain code, in board-specific units
- property nuller_railed: bool
True if nuller has recently hit a DAC rail
- property routing_type: str
Whether DAC are routed directly to ADCs or to the cryostat
- property squid_current_bias: float
SQUID current bias, in board-specific units (XXX)
- property squid_feedback: str
SQUID feedback mechanism employed
- property squid_flux_bias: float
Flux bias, in board-specific units (XXX)
- property squid_p2p: float
Peak-to-peak voltage difference in V-Phi curve stored by tuning script
- property squid_stage1_offset: float
Offset voltage applied to SQUID output before first-stage amp
- property squid_state: str
Descriptive string (e.g. “Tuned”) stored by control system tuning script to describe SQUID state
- property squid_transimpedance: float
Measured SQUID transimpedance, in resistance units
Class-like Pipeline Modules
Classes that can be added to G3Pipeline
instances, either written as callable objects in Python (see
Callable Objects as Functions) or as G3Module
-derived
classes in C++ (see Writing a module in C++). Aside from adding these into
pipelines using the standard Add()
method, such objects may also be
instantiated outside of pipelines to make use of additional features.
One typical use case is to extract some kind of data from a series of frames:
class ExtractData:
def __init__(self, key="RawTimestreams"):
self.key = key
self.data = []
def __call__(self, frame):
if self.key in frame:
self.data.append(frame[self.key])
# instantiate caching class
extractor = ExtractData()
# use instance in a pipeline
pipe = core.G3Pipeline()
pipe.Add(core.G3Reader, filename="file.g3")
pipe.Add(extractor)
pipe.Run()
# use extracted data in later processing
print(extractor.data)
Another use case may be to call additional methods of a class. For
example, the G3Writer
class can be used as a
context manager, and also report the position of the file pointer after
each frame is written:
with core.G3Writer("file.g3") as writer:
for frame in frames:
writer(frame)
print("Bytes written:", writer.tell())
Class-like modules permit a variety of additional features beyond their standard usage in pipelines.
- class spt3g.dfmux.DataQualityTools.FillMissingTimepointFrames[source]
Search for Timepoint frames that come at larger-than-expected intervals. Fill in the missin frames with garbage, and mark them as garbage for conversion to NaNs later.
- class spt3g.dfmux.DfMuxBuilder(boards: int, collation_tolerance: int = 1000000)
- class spt3g.dfmux.DfMuxBuilder(boards: IntVector, collation_tolerance: int = 1000000)
Bases:
G3EventBuilder
,G3Module
Processing module for data from DfMux boards. Reads data from boards data acquisition boards, requiring that data from all be timestamped to within collation_tolerance (default 10 microseconds) to be considered part of a single sample. If boards is an integer, listens for that number. If a list of integers, DfMuxBuilder will filter for only boards with serial numbers in the list.
- class spt3g.dfmux.DfMuxCollator(
- flac_compress: bool = True,
- drop_timepoints: bool = True,
- record_sampletimes: bool = True,
Bases:
G3Module
Collects DfMux timepoints into scan frames using a provided wiring map. Scan frames are created when an empty Scan frame appears in the data stream. This frame will contain all subsequent timepoints until either the next Scan frame is detected or the stream ends. In addition to dfmux timestreams, any scalar floating numbers that recur in every input Timepoint frame will be combined into a G3Timestream of the same name stored in the output scan frame.
- class spt3g.dfmux.HardwareMapTools.GenerateFakeHardwareMap(frame)[source]
Inserts a fake hardware map into the data stream. Takes a wiring frame as an argument to the constructor, which it will inject before any other frames.
- class spt3g.dfmux.HardwareMapTools.PyDfMuxWiringMapInjector(pydfmux_hwm, pathstring=None, state=[])[source]
Insert a wiring map derived from a pydfmux hardware map into the data stream ahead of what would otherwise be the first frame.
Optionally filter for detectors described by the mask in <pathstring> (see pydfmux documentation for hwm.channel_maps_from_pstring()) and detectors in one of the states identified by the state argument.
- class spt3g.dfmux.Housekeeping.HousekeepingConsumer(ignore_wiring=False)[source]
Collect housekeeping data from the mux boards defined in the wiring map. Will add a key called ‘DfMuxHousekeeping’ to any housekeeping frame that goes by containing the data as of the arrival of the housekeeping frame. Use in conjunction with a dfmux.PeriodicHousekeepingCollector to get data at fixed intervals.
Also emits a Wiring frame from the housekeeping data. This requires a recent (as of November 2018) version of pydfmux in order to read mapped channel names from each board. If ignore_wiring=False, assumes that the wiring map is handled by a separate process.
If collecting real-time data, you may want to set subprocess=True when adding this module.
- classmethod HousekeepingForBoard(hostname)[source]
Return HkBoardInfo object from a board. Useful for debugging.
- classmethod HousekeepingFromJSON(dat)[source]
Build HKBoardInfo object from a JSON blob returned by the _dump_housekeeping call
- classmethod WiringFromJSON(dat, ip, crate, slot)[source]
Build WiringMap object from a JSON blob returned by the _dump_housekeeping call
- class spt3g.dfmux.Housekeeping.PeriodicHousekeepingCollector(N=15200)[source]
Inserts housekeeping frames every N timepoints.
- class spt3g.dfmux.LegacyHousekeeping.LegacyHousekeepingConsumer[source]
Collect housekeeping data from the legacy (i.e. SPTpol-ish) mux boards defined in the wiring map. Will add a key called ‘DfMuxHousekeeping’ to any housekeeping frame that goes by containing the data as of the arrival of the housekeeping frame. Use in conjunction with a dfmux.PeriodicHousekeepingCollector to get data at fixed intervals.
If collecting real-time data, you may want to set subprocess=True when adding this module.
- class spt3g.dfmux.NetCDFDump(filename: str)
Bases:
G3Module
Writes DfMux streamer data to a NetCDF file
- class spt3g.dfmux.unittransforms.ConvertTimestreamUnits(
- Input='RawTimestreams',
- Output='CalTimestreams',
- Units=spt3g.core.G3TimestreamUnits.Power,
- SkipUncalibratable=False,
- KeepConversionsForObservation=True,
Changes timestream units from one set of units (e.g. ADC counts) to another (e.g. Power). Currents and power are time averaged quantities (i.e. currents give RMS values).
Note that this does not handle conversions to on-sky quantities (e.g. K_cmb)
Copy data in the timestream map Input to the timestream map Output, converting the units from whatever they were to those specified by Units.
If SkipUncalibratable is true, copy timestreams for which the unit conversions could not be evaluated into the output timestream in their original units. If false, throws an exception if this occurs.
If KeepConversionsForObservation is True (default), conversion factors will only be evaluated once per observation. Note that this will cause the wrong conversions to be applied if any parameters of the mux system are modified during the observation but will substantially increase performance.
Function-like Pipeline Modules
Python functions that can be added to G3Pipeline
instances. Such functions may also be called directly with a G3Frame
object as the first argument, and do not necessarily need to be used in a pipeline. See Python Modules as Functions for more detail.
- spt3g.dfmux.HardwareMapTools.PyDfMuxBolometerPropertiesInjector(
- frame,
- pydfmux_hwm=None,
- angle_per_mm=7.305948248848263e-05,
Insert a calibration frame following any wiring frame containing a BolometerPropertiesMap named “NominalBolometerProperties” that has the properties of each bolometer as defined by the given pydfmux hardware map.
Useful Classes
Various Python and C++ classes that are part of the public API.
- class spt3g.dfmux.DfMuxCollector(
- builder: G3EventBuilder,
- hostnames: StringVector,
- class spt3g.dfmux.DfMuxCollector(
- interface: str,
- builder: G3EventBuilder,
- boardlist: IntVector = spt3g.core.IntVector([]),
- class spt3g.dfmux.DfMuxCollector(
- interface: str,
- builder: G3EventBuilder,
- board_serial_map: dict,
Listener object that collects IceBoard packets from a single network interface and decodes and forwards them to a DfMuxBuilder object for insertion into the data stream. Takes the builder object to which the data should be sent. Uses either multicast UDP or SCTP depending on which constructor is used.
Signature 1: Create a DfMuxCollector listening for SCTP packets from the listed hosts (e.g. [“iceboard0062.local”, …]) and forwards it to DfMuxBuilder “builder”.
Signature 2: Create a DfMuxCollector listening on “interface” for multicasted UDP packets and forwards it to DfMuxBuilder “builder”. Filters to only the boards specified in “boardlist” (by default empty, implying all boards).
Signature 3: Crate a DfMuxCollector that can parse V2 (64x) data. Pass a mapping from board IP address (strings or integers) to serial numbers as the last argument
- Start() int
- Stop() int
- property clock_rate: float
Set the clock rate for the iceboard subseconds counter, e.g. for hidfmux. Values should be in G3Units of frequency. Defaults to 100*core.G3Units.MHz.
- class spt3g.dfmux.HkChannelInfoMap()
- class spt3g.dfmux.HkChannelInfoMap(arg: HkChannelInfoMap)
- class spt3g.dfmux.HkChannelInfoMap(arg: Iterable)
Mapping of channel number (1-indexed) to channel status information
Signature 2: Copy constructor
Signature 3: Iterable constructor
- clear()
Remove all items from the mapping.
- copy() HkChannelInfoMap
Return a shallow copy of the mapping.
- get(key: int, default: object = None) object
Return the value for key if key is in the mapping, else default.
- pop(key: int) HkChannelInfo
- pop(key: int, default: object) object
Signature 1: Remove specified key and return the corresponding value. If the key is not found, raise a KeyError
Signature 2: Remove specified key and return the corresponding value. If the key is not found, return the default
- update(items: Iterable = [], **kwargs)
Update mapping from iterable/mapping.
- values() ValuesView
Return an object providing a view on the mappings’s values.
- class spt3g.dfmux.HkMezzanineInfoMap()
- class spt3g.dfmux.HkMezzanineInfoMap(arg: HkMezzanineInfoMap)
- class spt3g.dfmux.HkMezzanineInfoMap(arg: Iterable)
1-indexed mapping of mezzanine ID to mezzanine-specific housekeeping data
Signature 2: Copy constructor
Signature 3: Iterable constructor
- clear()
Remove all items from the mapping.
- copy() HkMezzanineInfoMap
Return a shallow copy of the mapping.
- get(key: int, default: object = None) object
Return the value for key if key is in the mapping, else default.
- pop(key: int) HkMezzanineInfo
- pop(key: int, default: object) object
Signature 1: Remove specified key and return the corresponding value. If the key is not found, raise a KeyError
Signature 2: Remove specified key and return the corresponding value. If the key is not found, return the default
- update(items: Iterable = [], **kwargs)
Update mapping from iterable/mapping.
- values() ValuesView
Return an object providing a view on the mappings’s values.
- class spt3g.dfmux.HkModuleInfoMap()
- class spt3g.dfmux.HkModuleInfoMap(arg: HkModuleInfoMap)
- class spt3g.dfmux.HkModuleInfoMap(arg: Iterable)
Mapping from 1-indexed module numbers to module-specific housekeeping data
Signature 2: Copy constructor
Signature 3: Iterable constructor
- clear()
Remove all items from the mapping.
- copy() HkModuleInfoMap
Return a shallow copy of the mapping.
- get(key: int, default: object = None) object
Return the value for key if key is in the mapping, else default.
- pop(key: int) HkModuleInfo
- pop(key: int, default: object) object
Signature 1: Remove specified key and return the corresponding value. If the key is not found, raise a KeyError
Signature 2: Remove specified key and return the corresponding value. If the key is not found, return the default
- update(items: Iterable = [], **kwargs)
Update mapping from iterable/mapping.
- values() ValuesView
Return an object providing a view on the mappings’s values.
- class spt3g.dfmux.LegacyDfMuxCollector(
- port: int,
- builder: G3EventBuilder,
- mcastlistenaddr: str = '',
- mcastgroupaddr: str = '',
Listener object that collects legacy network packets and decodes and forwards them to a DfMuxBuilder object for insertion into the data stream. Takes a builder object to which the data should be sent. If boards running in multicast code, also takes the address of the interface on which to listen and the multicast group address. Only works with DAN streamers.
- Start() int
- Stop() int
Useful Functions
Various Python and C++ functions that are part of the public API.
- spt3g.dfmux.HardwareMapTools.PathStringForBolo(wiringmap, bolo)[source]
Obtain the channel pathstring for a bolometer named “bolo” using the passed wiring map.
- spt3g.dfmux.Housekeeping.HousekeepingForBolo(hkmap, wiringmap, bolo, all_hk=False)[source]
Obtain the channel housekeeping information for a bolometer named “bolo” using the passed housekeeping and wiring maps.
If all_hk is True, returns a tuple of the (board, mezz, module, channel) HK data instead of just the channel.
- spt3g.dfmux.unittransforms.get_timestream_unit_conversion(
- from_units,
- to_units,
- bolo,
- wiringmap=None,
- hkmap=None,
- system=None,
- tf=None,
Return the scalar conversion factor to move timestream data from a given system of units (Power, Resistance, Current, Counts) to another one. Requires a wiring map and recent housekeeping data. Returned quantities are RMS for currents and time-averaged for power.
Note that this does not handle conversions to on-sky quantities (e.g. K_cmb)