The Endplug High Voltage System:

Control & Monitoring

User Manual – V1.2

15 February 2001

Luciana.Malferrari@bo.infn.it – University and INFN-Bologna

 

Introduction

The Endplug High Voltage System is made of 12 power supply crates type CAEN SY527, housing a total of about 3000 high voltage channels, connected to three sub-detectors: the Electromagnetic Calorimeter, the Hadron Calorimeter and the Shower Max. Crates are remotely programmed and controlled by programs running on a WindowsNT4/PC. All crates are chained together by a serial line and connected to the control PC via the A303 interface. The PC node is cdfephv and it is located at the 3rd floor-B0. Two programs perform HV control and monitoring: the first, named hvServer, drives the I/O with the interface (type A303) connected to the CAEN serial line; the second, named hvClient, represents the User-Interface to the HV System.

Paragraphs 1,2,3 are necessary to know the structure of executable and auxiliary files, from paragraph 4.2 to 4.9 expert operations and relevant panels are explained while paragraph 4.2 contains the information for monitoring and on/off operations.

The global scheme of the Endplug High Voltage System is shown in Fig.1.

 

Figure 1

 

 

  1. Software components

The cdfephv operating system is Windows NT 4.0 Workstation. The hvServer.exe and hvClient.exe programs are located under E:/hv which represents the default HV-System working folder. Both programs should be started automatically at the operating system startup, however at the moment they are started manually from the working folder or from the usual NT Start->Program command bar. A backup copy of both programs is placed under the backup folder Q:/hvBackup and can be started in case of corruption of files in disk E. Each process opens a standard MS-DOS type window which represents the process log screen. Messages of both log screens are also saved on separate logfiles, named hvServer-mm-dd-yyyy.log and hvClient-mm-dd-yyyy.log, where mm-dd-yyyy correspons to the current date. If the date changes while a process is running, a new log file, with a new name, is automatically opened. The logfile is written in text format (can be viewed by double clicking on it), and saved into the default folder Q:/hvLog.

Three more files, used by the hvClient process, are necessary:

More details about the Startup and the Configuration file are given in Appendix A and B respectively.

 

  1. HV-Server
  2. The hvServer process is unique and has to be running on cdfephv since this is the only PC connected to the CAEN serial line. The server process must be started before any client process. When the server is not running, for example during a PC reboot, the access to any CAEN power supply is impossible through the serial line. In this unfortunate situation some emergency commands to the high voltage crates are still possible from remote. These commands are realized by hardware signals, provided by a homemade TTL generator, which are fed to each CAEN crate through some RG 52 cables. A description of such signals is given in section 4.

  3. HV-Client

The hvClient process can run on any NT Workstation system and there is no limitation to the number of client processes. This allows any Endplug HV expert to get a connection to the HV system without overloading cdfephv. The client performs two main functions:

Control operations, usually demanded to an Endplug HV expert, require a password to be entered.

At the startup, a properly operating hvClient loads three files and establishes a connection with the hvServer process, according to the following order:

The Configuration present in the Configuration file normally matches the one already loaded onto the CAEN systems. The loading of a Configuration (parameters and groups) onto the power supplies takes about some minutes per crate, and should be performed only under the following two circumstances:

The configuration parameters loaded on each Crate are stored on EEPROMs devices, this means that data are permanent even when the power supply is switched off.

  1. User-Interface
  2. The User Interface is started by any hvClient process and consists of graphical panels containing menu, command buttons, text boxes, indicators etc. The Main Menu panel (Fig.2) is automatically displayed, together with the Detector ON/OFF& Status panel (Fig.3), at the process startup. Other panels can be opened by the commands of the Main Menu panel. Note that not all menu commands are available at the same time since some operations are not allowed to overlap. Menu commands marked with (*) are not implemented yet.

     

    Figure 2

     

     

    Figure 3

     

     

     

     

     

     

     

     

     

     

     

    1. Main Menu panel

The items of the six menus contained in the Main Menu (Fig.2) are here expaned and labeled in the case they open a new specific panel.

 

 

 

 

 

4.2 HV Monitoring: Detector ON/OFF & Status panel

This panel (Fig.3) is automatically opened at the process startup. It contains command buttons to switch detectors On/Off. and indicators of the global status of detectors and crates. A detector or crate indicator can be clicked to obtain a detailed status of the corresponding object. The color of an indicator corresponds to the global status for that object and can be:

Green = Okay; Yellow = Warning; Red = Alarm.

The panel also shows the status of the Monitor process, which can be in the Running, Paused or Stopped status. The information shown by indicators can be updated only if the Monitor process is Running. The Operator can control the status of the Monitor process with the commands Start, Stop, Pause, Resume, available in Main Menu -> HV Monitoring. Since the monitoring cycle is executed in three steps, one step for each detector, when the Operator asks the Monitor to pause, the current step (i.e. detector) is completed. Instead, when the Operator asks the Monitor to stop, the whole monitoring cycle (i.e. all three detectors) is completed before stopping.

When a given detector is switched On or Off, the wanted voltage status becomes the "Expected Status" for that detector and the Monitor takes that as a reference to declare (in future) the current status of the detector. For example a detector that has been switched On by the Operator is "Expected to be On" and the corresponding HV-Status indicator will be set "On" (green) if all related channels will be found (by the Monitor) with state On, at the expected voltage.

The "Expected Status" is automatically saved on the HV-Startup file whenever a detector is switched Off or On. This allows recovering the last "Expected Status" of all detectors, in case of a client restart (or crash). In some circumstances, for example after a power cut or when the voltage status is modified by more than one hvClient process (the latter should never happen), it could be that the "Expected Status" (loaded from the HV-Startup file) doesn’t match the real status. The Operator can easily fix this at the startup after switching the detectors to the wanted state.

The list of possible Detector status, filled in the HV-Status indicator, is:

Equivalent Status indicators are available for each Crate also with some more information like:

The monitoring parameters, like the sleep-time between two monitoring cycles, the threshold that defines a channel to be under-voltage or over-voltage, can be set in the Main Menu -> HV Monitoring -> Monitoring Parameters.

In the Main Menu -> HV Monitoring -> Ignore Channels the Operator can flag a channel, a module or a whole crate, as Ignored. The Ignore flag is useful for example when a channel is known to have some problem and the Operator wants to avoid the injection of warning or error messages. At the process startup or after the loading in memory of a new configuration, some diagnosis of the system is automatically performed.

The complete system diagnosis will take some time since it includes the following checks:

Since the Monitor process is started in multithread mode, any other operation can be performed in parallel on the system. Exceptions are the operations that could affect the Configuration in use. Those operations are disabled while the Monitor is running. For example the Operator has to stop the Monitor before loading a new configuration in memory or before defining new groups of channels or before setting the Ignore flag for some channel.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

4.3 HV Monitoring: Monitoring Parameters panel

 

 

Figure 4

This panel is opened through the Main Menu -> HV Monitoring -> Monitoring Parameters.

The panel allows to see or to modify some monitoring parameters:

Before modifying any parameter the Operator has to manually pause (and later resume) the Monitor to be allowed to perform such a modification. The new configuration becomes effective immediately and it will be automatically saved (not yet implemented) on the HV-Startup file, before exiting the panel.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

4.4 HV Monitoring: Ignore Channels panel

 

Figure 5

This panel is opened through the Main Menu -> HV Monitoring -> Ignore Channels.

This panel is intended for Endplug HV experts only and allows see or to modify the Ignore flag for a channel, a module or a whole crate. Before modifying (set or reset) any Ignore flag the Operator has to manually Pause (and later Resume) the Monitor to be allowed to perform such a modification. The new configuration becomes effective immediately and it will be asked a save on a Configuration file, before exiting the panel. The save is not mandatory but in case of a client restart (or crash) the last modifications to the Ignore flag configuration would be lost.

 

4.5 System Configuration: Group Map panel

 

 

Figure 6

This panel is opened through the Main Menu -> System Configuration -> Group Map.

The panel allows to show and to modify the channel-to-group assignment and is intended for

Endplug HV experts only.

The CAEN SY527 allows to group channels under the same group ID, up to 16 different IDs. Such a grouping is useful especially for monitoring or switching Off/On entities like detectors. In fact, a unique read/write can perform a given operation on all channels belonging to the same group. In our system the channel grouping is made according to the channel-to-detector assignment. The channel grouping is permanently loaded on each CAEN system according to the Configuration file. The group assignment is a rather delicate point and it should never be modified without contacting an expert having full knowledge of the hardware and software system.

 

4.6 System Configuration: Cabling Map panel

 

Figure 7

This panel is opened through the Main Menu -> System Configuration -> Cabling Map.

The panel allows seeing the HV-channel to Detector-towers cabling for each CAEN system. The cabling map is the one loaded from the Configuration file. For each HV-channel it is possible to see the correspondence to the Tower, Wedge, PMT-box, Detector name and Detector end. It is also possible to see which HV-channels are left unconnected (i.e. unused). In case of physical or just a "formal" modification of the cabling map, the same modification must be introduced in the Configuration file accordingly.

 

4.7 Diagnostic: Search-Log-messages panel

Figure 8

 

4.7 HV Operations: Operation-by-Channel panel

 

Figure 9

This panel is opened through the Main Menu -> HV Operation -> Set/Show by Channel.

The panel allows, for a given channel (in a given module and Crate), to switch the voltage Off/On, to read or to modify the configuration parameters loaded in the CAEN crate.

This panel is intended for Endplug HV experts only.

Before modifying any parameter or switching Off/On a given channel the Operator has to consider if that operation will cause a warning/alarm from the Monitor process and eventually

Pause (an later Resume) the Monitor Run before doing that operation. In case the Operator

desires to permanently modify (on the CAEN) some channel parameters (the Off/On switching

is not a channel parameter), such a modification can be done through this panel, after the

Monitor has been Paused or Stopped. The new configuration becomes effective immediately

and it will be asked a save on a Configuration file, before exiting the panel. The save is not

mandatory but in case of a client restart (or crash) the modified CAEN configuration would

not match the Configuration file and some monitoring warning/error condition could appear.

When the same parameter modification has to be applied to all channels of the same detector

the Operator could take advantage by doing that operation through the Operation-by-Group panel.

 

 

 

 

 

4.8 HV Operations: Operation-by-Group panel

 

 

Figure 10

This panel is opened through the Main Menu -> HV Operation -> Set/Show by Group.

This panel is intended for Endplug HV experts only.

The panel allows, for a given group of channels, to switch Off/On or to read or to modify the configuration parameters loaded on CAEN systems. The Group name is the first item to be specified entering the panel. The specified Group of channels will be searched for on all crates (according the current configuration) of the system. Before modifying any parameter or switching Off/On a given group of channels the Operator has to consider if that operation will cause a warning/alarm from the Monitor process and eventually Pause (an later Resume) the Monitor Run before doing that operation. In case the Operator desires to permanently modify (on the CAENs) some channel parameters (the Off/On switching is not a channel parameter), such a modification can be done through this panel, after the Monitor has been Paused or Stopped. The new configuration becomes effective immediately and it will be asked a save on a Configuration file, before exiting the panel. The save is not mandatory but in case of a client restart (or crash) the modified CAEN configuration would not match the Configuration file and some monitoring warning/error condition could appear.

 

 

 

 

 

 

 

4.9 Channel History panel

This panel is opened through the Main Menu -> HV Monitoring -> Channel History (not implemented yet). This panel allows showing a graphical display the voltage (and current) history for any channel of the system. The history is saved, and continuously updated, for a given channel if a History flag has been set for that channel. The History flag is automatically generated by the Monitor process for channels having out of order parameters. Moreover the Operator through this panel can manually set the History flag for any channel.

The history is updated only if the Monitor is in running state. Putting the Monitor into a paused state will generate holes in the channel history.

5 The TTL generator

6 Problems, traps and trick

App A: HV-Startup file

* *********************************************************************

* File name: HV-Startup.txt

*

* Startup file for the End Plug HV control&monitoring Instructions:

* this file contains 12 meaningful items !

* Any line starting with "*" in column 1 is a comment line.

* The items' order cannot be modified.

* Please insert new items before last three lines (detector status).

* *********************************************************************

*

*Item 1: LogFile Directory

*

F:\HvLog

*

*Item 2: IP address of the HV-Server node

*

cdfephv.fnal.gov *Fnal

*

*Item 3: Default Configuration File name

*

FCrate0112.txt

*

*Item 4: Caen Interface available/not-available: 1/0

*

* 0 *A303 is not present

1 *A303 is present

*

*Item 5: Fix Writing switched on/off: 1/0

*

0 *Fix writing switched off

* 1 *Fix writing switched on

*

*Item 6: Check Communication with Hardware, at the startup, switched on/off: 1/0

*

* 0 * Check Communication with Hardware switched off

1 *Check Communication with Hardware switched on

*

*Item 7: Check Group consistency of CAEN vs Configuration, at the startup, switched on/off: 1/0

*

0 * Check Group consistency of CAEN vs Configuration switched off

* 1 *Check Group consistency of CAEN vs Configuration switched on

*

*Item 8: Check Parameters consistency of CAEN vs Configuration, at the startup, switched on/off: 1/0

*

0 * Check Parameters consistency of CAEN vs Configuration switched off

* 1 *Check Parameters consistency of CAEN vs Configuration switched on

*

*Item 9: AutoPilotMonitor, one monitor cycle at the startup, switched on/off: 1/0

*

0 *AutoPilotMonitor switched off

* 1 *AutoPilotMonitor switched on

*

* Item 10-12: next 3 lines contain the REQUIRED HV status for the detectors.

* Note: no line can follow the last Detector status line

* Status is 0=OFF 1=ON.

EM_REQUIRED_ON 0

HA_REQUIRED_ON 0

SH_REQUIRED_ON 0

App. B: Configuration file

*Format3

*Crate01

* Gr 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Sl Ch VMax V0Set I0Set V1Set I1Set Rup Rdwn Trip Twr Wdg PMT Det Plug IgnoreFlag

1 1 0 1 0 0 0 1 0 1 0 0 0 1 0 0 0 00 1180.0 1180.0 00.000 0000.0 00.000 100 100 1000 -1 23 01 PHa West 0

1 1 0 1 0 0 0 1 0 1 0 0 0 1 0 0 0 01 1180.0 1180.0 00.000 0000.0 00.000 100 100 1000 -1 23 01 PHa West 0

1 1 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 02 1033.0 1033.0 00.000 0000.0 00.000 100 100 1000 02 23 01 PHa West 0

1 1 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 03 1231.0 1231.0 00.000 0000.0 00.000 100 100 1000 03 23 01 PHa West 0

1 1 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 04 1101.0 1101.0 00.000 0000.0 00.000 100 100 1000 04 23 01 PHa West 0

1 1 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 05 1163.0 1163.0 00.000 0000.0 00.000 100 100 1000 05 23 01 PHa West 0

1 1 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 06 1153.0 1153.0 00.000 0000.0 00.000 100 100 1000 06 23 01 PHa West 0

1 1 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 07 1212.0 1212.0 00.000 0000.0 00.000 100 100 1000 07 23 01 PHa West 0

1 1 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 08 1080.0 1080.0 00.000 0000.0 00.000 100 100 1000 08 23 01 PHa West 0

1 1 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 09 1075.0 1075.0 00.000 0000.0 00.000 100 100 1000 09 23 01 PHa West 0

1 1 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 10 1068.0 1068.0 00.000 0000.0 00.000 100 100 1000 10 23 01 PHa West 0

1 1 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 11 1110.0 1110.0 00.000 0000.0 00.000 100 100 1000 11 23 01 PHa West 0

1 1 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 12 1107.0 1107.0 00.000 0000.0 00.000 100 100 1000 12 23 01 PHa West 0

1 1 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 13 1082.0 1082.0 00.000 0000.0 00.000 100 100 1000 13 23 01 PHa West 0

1 1 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 14 1072.0 1072.0 00.000 0000.0 00.000 100 100 1000 14 23 01 PHa West 0

1 1 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 15 1096.0 1096.0 00.000 0000.0 00.000 100 100 1000 15 23 01 PHa West 0

1 1 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 16 1149.0 1149.0 00.000 0000.0 00.000 100 100 1000 16 23 01 PHa West 0

1 1 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 17 1081.0 1081.0 00.000 0000.0 00.000 100 100 1000 18 23 01 PHa West 0

1 1 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 18 1221.0 1221.0 00.000 0000.0 00.000 100 100 1000 20 23 01 PHa West 0

1 1 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 19 1035.0 1035.0 00.000 0000.0 00.000 100 100 1000 22 23 01 PHa West 0

1 1 0 1 0 0 0 1 0 1 0 0 0 1 0 0 0 20 1180.0 1180.0 00.000 0000.0 00.000 100 100 1000 -1 23 01 PHa West 0

1 1 0 1 0 0 0 1 0 1 0 0 0 1 0 0 0 21 1180.0 1180.0 00.000 0000.0 00.000 100 100 1000 -1 23 01 PHa West 0

1 1 0 1 0 0 0 1 0 1 0 0 0 1 0 0 0 22 1180.0 1180.0 00.000 0000.0 00.000 100 100 1000 -1 23 01 PHa West 0

1 1 0 1 0 0 0 1 0 1 0 0 0 1 0 0 0 23 1180.0 1180.0 00.000 0000.0 00.000 100 100 1000 -1 23 01 PHa West 0

1 1 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 24 1281.0 1281.0 13.000 0000.0 13.000 100 100 1000 -1 23 01 PHa West 0

* Gr 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Sl Ch VMax V0Set I0Set V1Set I1Set Rup Rdwn Trip Twr Wdg PMT Det Plug IgnoreFlag

1 1 0 1 1 0 1 0 0 0 0 0 0 0 0 0 1 00 0818.0 0818.0 00.000 0000.0 00.000 100 100 1000 00 23 01 PEl West 0

1 1 0 1 1 0 1 0 0 0 0 0 0 0 0 0 1 01 0824.0 0824.0 00.000 0000.0 00.000 100 100 1000 01 23 01 PEl West 0

1 1 0 1 1 0 1 0 0 0 0 0 0 0 0 0 1 02 0848.0 0848.0 00.000 0000.0 00.000 100 100 1000 02 23 01 PEl West 0

1 1 0 1 1 0 1 0 0 0 0 0 0 0 0 0 1 03 0825.0 0825.0 00.000 0000.0 00.000 100 100 1000 03 23 01 PEl West 0

1 1 0 1 1 0 1 0 0 0 0 0 0 0 0 0 1 04 0827.0 0827.0 00.000 0000.0 00.000 100 100 1000 04 23 01 PEl West 0

1 1 0 1 1 0 1 0 0 0 0 0 0 0 0 0 1 05 0855.0 0855.0 00.000 0000.0 00.000 100 100 1000 05 23 01 PEl West 0

1 1 0 1 1 0 1 0 0 0 0 0 0 0 0 0 1 06 0801.0 0801.0 00.000 0000.0 00.000 100 100 1000 06 23 01 PEl West 0

1 1 0 1 1 0 1 0 0 0 0 0 0 0 0 0 1 07 0820.0 0820.0 00.000 0000.0 00.000 100 100 1000 07 23 01 PEl West 0

1 1 0 1 1 0 1 0 0 0 0 0 0 0 0 0 1 08 0849.0 0849.0 00.000 0000.0 00.000 100 100 1000 08 23 01 PEl West 0

1 1 0 1 1 0 1 0 0 0 0 0 0 0 0 0 1 09 0793.0 0793.0 00.000 0000.0 00.000 100 100 1000 09 23 01 PEl West 0

1 1 0 1 1 0 1 0 0 0 0 0 0 0 0 0 1 10 0878.0 0878.0 00.000 0000.0 00.000 100 100 1000 10 23 01 PEl West 0

1 1 0 1 1 0 1 0 0 0 0 0 0 0 0 0 1 11 0811.0 0811.0 00.000 0000.0 00.000 100 100 1000 11 23 01 PEl West 0

1 1 0 1 1 0 1 0 0 0 0 0 0 0 0 0 1 12 0852.0 0852.0 00.000 0000.0 00.000 100 100 1000 12 23 01 PEl West 0

1 1 0 1 1 0 1 0 0 0 0 0 0 0 0 0 1 13 0866.0 0866.0 00.000 0000.0 00.000 100 100 1000 13 23 01 PEl West 0

1 1 0 1 1 0 1 0 0 0 0 0 0 0 0 0 1 14 0845.0 0845.0 00.000 0000.0 00.000 100 100 1000 14 23 01 PEl West 0

1 1 0 1 1 0 1 0 0 0 0 0 0 0 0 0 1 15 0832.0 0832.0 00.000 0000.0 00.000 100 100 1000 15 23 01 PEl West 0

1 1 0 1 1 0 1 0 0 0 0 0 0 0 0 0 1 16 0883.0 0883.0 00.000 0000.0 00.000 100 100 1000 16 23 01 PEl West 0

1 1 0 1 1 0 1 0 0 0 0 0 0 0 0 0 1 17 0815.0 0815.0 00.000 0000.0 00.000 100 100 1000 18 23 01 PEl West 0

1 1 0 1 1 0 1 0 0 0 0 0 0 0 0 0 1 18 0904.0 0904.0 00.000 0000.0 00.000 100 100 1000 20 23 01 PEl West 0

1 1 0 1 1 0 1 0 0 0 0 0 0 0 0 0 1 19 0886.0 0886.0 00.000 0000.0 00.000 100 100 1000 22 23 01 PEl West 0

1 1 0 1 1 0 1 0 0 0 0 0 0 1 0 0 1 20 0900.0 0900.0 00.000 0000.0 00.000 100 100 1000 -1 23 01 PEl West 0

1 1 0 1 1 0 1 0 0 0 0 0 0 1 0 0 1 21 0900.0 0900.0 00.000 0000.0 00.000 100 100 1000 -1 23 01 PEl West 0

1 1 0 1 1 0 1 0 0 0 0 0 0 1 0 0 1 22 0900.0 0900.0 00.000 0000.0 00.000 100 100 1000 -1 23 01 PEl West 0

1 1 0 1 1 0 1 0 0 0 0 0 0 1 0 0 1 23 0900.0 0900.0 00.000 0000.0 00.000 100 100 1000 -1 23 01 PEl West 0

1 1 0 1 1 0 1 0 0 0 0 0 0 0 0 0 1 24 1007.0 1007.0 13.000 0000.0 13.000 100 100 1000 -1 23 01 PEl West 0

 

App. C: Simple instructions for 'non expert'

For most of the CAEN setting operations a password is required as a basic protection, but non-expert people can acess to monitoring information and switching On/Off the detectors.

  1. To Monitor Voltages:
  2. To Switch Channels On/Off: