RESTART PROCEDURES FOR TOF RATES/TDC SPECTRA Main changes since last year: 1) Processes on ararat and skyros RIO2 CPUs in rack M6 and O4 start automatically, nothing should be done manually, exept optional restarting of TofMon and SpikeHunter. 2) PTL2000 spike server/trigger unused now. 3) tofmacproxy process is added to h1tofpc2. The software which enables display of TOF/VETO rates on web ( http://h1tofpc2.desy.de) and on Lumi applet (via h1lumiserver, which sends also data to HERA via Nextmex), rates and TDC spectra in TofMon consist of the following: 1) rate server: process running on VME single board computer skyros.desy.de in rack O4. The process is started automatically at the VME crate power on. Please restart TofMon on h1tofpc2 after restart of the server. "lograte" process should restart automatically (to check whether "lograte" check last update time of rate history files: do on h1tofpc2 "ls -ltr /x01/usr/h1tof/rate/ratehist | tail"). To check whether processes are running use: logon as vdodo on skyros.desy.de ps x | grep rate2 | grep -v grep (there should be N_connections + 2 processes with name rate2) To stop rate server: logon as vdodo on skyros.desy.de ps x (note "pgrp" of processes with name ./rate2) kill pgrp (or kill -9 PIDs) Wait 5 seconds Check rate2 processes again: "ps x", if there are still processes with name rate2 then "kill -9" all of them. To start rate_server: logon as vdodo on skyros.desy.de . ~/rate/RUN2 Make sure that within few minutes Lumi Java client connects to the server: run "netstat" command and look for the line "tcp 0 0 skyros.desy.de.8010 h1lumiserver.des.53906 ESTABLISHED" If there are no connection then call Lumi expert (Igor Cheviakov tel. 3404) and ask him to restart client, otherwise "e-Bkgd" and "p-Bkgd" curves on Lumi display in Room 301 and in HERA control room will be frozen. 2) (unused now) p-spike server: process running on VME single board computer skyros.desy.de in rack O4. This is _unused_ at the moment. The process can be started manually after the VME crate was switched off. Please restart "dumpsike5" process on h1tofpc2 after restart of the server (if you really need it). To check whether processes are running use: logon as vdodo on skyros.desy.de ps x | grep rate5 | grep -v grep (there should be N_connections + 2 processes with name rate4) To stop spike server: logon as vdodo on skyros.desy.de ps x (note "pgrp" of processes with name ./rate4) kill pgrp (or kill -9 PIDs) Wait 5 seconds Check rate5 processes again: "ps x", if there are still processes with name rate5 then "kill -9" all of them. To start spike server: logon as vdodo on skyros.desy.de cd ~/spike ./ptl +ptl_al1.cfg cd ../newspike . RUN5 3) TDC/PTL server: process running on VME single board computer (ararat.desy.de) in rack M6. The process should start automatically during boot of RIO2 CPU after power on. To check whether processes are running: logon as vdodo on ararat.desy.de ps x | grep tdcsrv | grep -v grep (there should be N_connections + 2 processes with name tdcsrv) To stop server: logon as vdodo on ararat.desy.de ps x (note "pgrp" of processes with name ./tdcsrv) kill pgrp (or kill -9 PIDs) Wait 5 seconds Check tdcsrv processes again: "ps x", if there are still processes with name tdcsrv then "kill -9" all of them. To start server: logon as vdodo on ararat.desy.de login bma . ~/tdcsrv/RUN 4) FNC trigger elements FNC trigger elements are sent to Central Trigger via one PTL2000 module in VME crate in rack M6. The PTL2000 is programmed automatically during boot of RIO2 CPU after crate is powered on. If you still want to load the PTL2000 manually: logon as vdodo on ararat.desy.de ~/fnc/ptl_load 5) Nextmex client which maintains HERA beam currents in shared memory which is used by "lograte". The process name is mexcln, it should be running on h1tofpc2. To check whether process is running use: ps x | grep mexcln | grep -v grep You can run also ./showcurr (exit via Ctrl-C) -- the program shows beam currents in shared memory). The currents/lifetime should be updating. To start beam currents client: login as h1tof on h1tofpc2 cd /x01/usr/h1tof/nextmex nohup mexcln 0 0 1 /dev/null 2>&1 & 7) (unused now) p-spike logging process: dumpspike5, running on h1tofpc2 (if really needed). To check whether process is running use: login as h1tof on h1tofpc2 ps x | grep dumpspike | grep -v grep To stop spike logging process: login as h1tof on h1tofpc2 killall dumpspike5 (or killall -9 dumpspike5) To start spike logging process: goto the room 301 in North Hall locate TOF PC at A3RT Switch to the 2nd KDE panel: press Ctrl-F2 Press Ctrl-C in the xterm window cd ~/spike . DUMPSPIKE Switch back to 1st KDE panel: press Ctrl-F1 8) TDC setup program. Should be run once for each TDC once VME crate in M6 powered on. 1) login on ararat 2) cd tdcsrv 3) ./tdcreset ; ./tdcreset 1 Now it is made automatically during ararat's boot, no hand work needed. But _important_: TDC setup should be done only when CDAQ run is stopped or NIM crate in M6 switched off. So before switching on VME crate in M6 switch off NIM crate in M6, switch on VME crate, wait until CPU boots and initializes TDCs and PTLs and after that switch on NIM crate. 9) High resolution BTOF rate server. Started automaticaly during VME CPU ararat in M6 boot procedure. Process name is scalsrv. After (re)starting scalsrv restart SpikeHunter on h1tofpc2. To check whether processes are running use: logon as vdodo on skyros.desy.de ps x | grep scalsrv | grep -v grep (there should be N_connections + 3 processes with name scalsrv) To stop: logon as vdodo on skyros.desy.de ps x (note "pgrp" of processes with name scalsrv) kill pgrp (or kill -9 PIDs) Wait 5 seconds Check scalsrv processes again: "ps x", if there are still processes with name scalsrv then "kill -9" all of them. To start: login as vdodo on ararat and execute: cd /home/vdodo/hirscal . RUN 10) dumpscal process, which writes the high-resolution BTOF rate data to disk for further analysis (spikes). If needed you can start it. To check whether process is running use: login as h1tof on h1tofpc2 ps x | grep dumpscal | grep -v grep To stop process: login as h1tof on h1tofpc2 killall dumpscal (or killall -9 dumpscal) To start process: login as h1tof on h1tofpc2 cd ~/scal . RUN 11) TOF Mac input parameters -- tofmacproxy process there are plans to switch off both old CTrig Mac and Branch 11. These are beeing used to obtain input online values like RunNr, RunState,Clock_Delay,Solenoid_Current, e.t.c.. I changed TVmonitor to obtain needed data from new CTrig server via an intermediate program running on TOF-PC (h1tofpc2). If e.g. after reboot of TOF Mac you feel that the parameters in the Main Window are not correct you can restart the intermediate program -- login as h1shift on h1tofpc2 and execute "killall -9 tofmacproxy" command. In a couple of minutes parameters on TV_Monitor should be ok. To check that data transfer is active execute "netstat -t" command, you should see several closing conection to h1tof1 like: tcp 0 0 h1tofpc2.desy.de:42945 h1tof1.de:gxs-data-port TIME_WAIT tcp 0 0 h1tofpc2.desy.de:42944 h1tof1.de:gxs-data-port TIME_WAIT tcp 0 0 h1tofpc2.desy.de:42946 h1tof1.de:gxs-data-port TIME_WAIT tcp 0 0 h1tofpc2.desy.de:42941 h1tof1.de:gxs-data-port TIME_WAIT tcp 0 0 h1tofpc2.desy.de:42943 h1tof1.de:gxs-data-port TIME_WAIT To start "tofmacproxy" after h1tofpc2 reboot: login as h1shift on h1tofpc2 and execute: " cd tofmac; nohup tofmacproxy.com & ". To look at data sent by CTrig use "./ctbunch" program in "~/tofmac" directory. After VME crate in O4 was powered off/on: 1) restart if necessary (kill then start) lograte (item 6 above) 2) restart TofMon in room 301 if PTL2000 spike logging is needed then additionally 3) stop dumpsike process (item 7 above) 4) start spike server (item 2 above) 5) start dumpsike process (item 7 above) After VME crate in M6 was powered off/on: 1) restart SpikeHunter in room 301 if needed 2) if needed (re)start dumpscal (item 10 above) After h1tofpc2 was rebooted/powered off: 1) start mexcln process (item 5 above) 2) start lograte process (item 6 above) 3) start tofmacproxy process (item 11 above) 4) Start TofMon in room 301 5) if needed, start dumpspike process (item 7 above) 6) start SpikeHunter in room 301 if needed 7) if needed start dumpscal (item 10 above) There are also several cron jobs which update preselected www views. They are started automatically. See "crontab -l" output. crontab file is in /x01/usr/h1tof/rate/crontab. The web server (apache) running on h1tofpc2 under root acount. Started automatically during PC boot. or ("rcapache stop; rcapache start" under root) Copy of the nominal config file for httpd -- /etc/httpd/httpd.conf is in /x01/usr/h1tof/rate/. To update local date/time on h1tofpc2 login as root and execute: /home/h1tof/netdate time.desy.de && hwclock --systohc <> Beam dump signal software working as follows: Counting rate value of each monitored channel (now BTS6_1, BTS6_2,BTS8_1,BTS8_2) averaged (exponentionally) over some time, when any of these averages becomes large than a limit beam dump signal is activated (Log. 1 on output). Beam dump signal is logical 0 if all averages are below limit. Update of averages and output signal switching if necessary is done after each readout cycle, i.e. with 0.5s period. Output signal is on 3rd from top NIM-Lemo output (named CMon Out). Dump request active is -0.8V, inactive 0V. Be carefull -- without 50Ohm load this CMon Out can give wrong voltages, another thing Lemo ground part can be not really grounded. Another thing -- when the system will be active be carefull with BTS6 and BTS8 single rates, if for example you put for tests 10MHz signal on BTS6_1 input of CFD then beam dump will be activated --> unintended beam loss. Beam dump monitoring is done within rate2 -- rate server program on skyros. Setting of channels to be monitored is specified in the end of ~vdodo/rate/rate2.cfg text file. Example: 0x311000 # VME base address of output PTL2000 0 # out polarity -- 0: alarm at log. 1, 1: at log. 0 0x10000 # debug level 4 # number of channels to check # mod_id chn_no tau limit 0x3801 11 30.0 5.0e6 # BTS6_1 0x3801 27 30.0 5.0e6 # BTS6_2 0x3801 12 30.0 5.0e6 # BTS8_1 0x3801 28 30.0 5.0e6 # BTS8_2 Here interesting things are: 0x10000 # debug level If this number is 0x10000 then debugging information about beam dump channels is put in unused now channels 9-15 of the last SIS3801 module. Channel 15 contains an AlarmCode -> divide this number by 100000, round it to integer, then resulting integer in the range 0-15 is a bitmask of channels in alarm state. Channels 9-14: value of averaged in time rate. If debug_level is 0 then SIS3801 reading are not overwritten, no debug information. 4 # number of channels to check The number of monitored channels, should be equal to number of channel parameter lines after this number. Monitored channel parameters lines # mod_id chn_no tau limit 0x3801 11 30.0 5.0e6 # BTS6_1 0x3801 27 30.0 5.0e6 # BTS6_2 0x3801 12 30.0 5.0e6 # BTS8_1 0x3801 28 30.0 5.0e6 # BTS8_2 Here "mod_id" "chn_no" pair define which rate to monitor. In the example all interestning channels are in SIS1 (see ~vdodo/h1/rate/rate_chmap.t). "Tau" is average time in seconds, "limit" is the limit after which alarm is generated. Parameters are defined so, that here for example, a constant rate of 5MHz apeared on a channel input will produce alarm in 30 seconds. After changing rate2.cfg file you need to restart rate server on skyros (see item 1 above) to make use of new parameters.