GlacioBasis Manual Revision 1 (3rd November 2009) Compiled by Michele Citterio, GEUS
Table of contents Table of contents.......................................................................................................... 1 Preface........................................................................................................................... 2 Revisions history......................................................................................................... 2 GlacioBasis ..................................................................................................
Preface This manual provides information required to properly carry out GlacioBasis glaciological fieldwork at the A.P. Olsen ice cap taking advantage of the logistic support provided by the Zackenberg Research Station. The focus is on the field procedures and technical details to be strictly followed to ensure the consistency and continuity of the data collection in the field. The manual in this first version should be regarded as being still in draft status.
GlacioBasis The primary aim of the GlacioBasis monitoring programme at Zackenberg is to produce a record of high quality glaciological observations from the A.P. Olsen Ice Cap and its outlet glacier in the Zackenberg river basin. The A.P. Olsen Ice Cap is located at 74° 39’ N and 21° 42’ W. The summit reaches an elevation of 1425 m and the terminus of the outlet glacier contributing to the Zackenberg river basin is at 525 m (figure 1).
importance of GlacioBasis monitoring tasks. The anticipated use of the monitoring data is to model the surface energy balance and the glacier mass budget with physically based models that, once calibrated and validated with in situ data, will allow to model the response of the glacier to future climate change scenarios. Figure 1 provides an overview and map of the investigated outlet glacier of the A.P. Olsen ice cap.
General instructions All GlacioBasis personnel arriving at Zackenberg station must have read the ZERO Site Manual (available from http://www.zackenberg.dk/NR/rdonlyres/8BA15D73-8D584C17-ABDA-99C0067FD571/0/ZERO20Site20Manual2C20ver2027102008.pdf) and must commit to follow its instructions. This is particularly important with regard to the restricted mobility in the protected areas around Zackenberg Station. Safety Safety is the first priority for all GlacioBasis personnel in the field.
for reference) is carried in the field. Use of special field notebooks, A4 size sheets and printed forms made with water-resistant paper is recommended. Electronic data will need to be duplicated on at least two separate storage supports to safeguard against data corruption.
Field procedures Ablation and velocity stakes A network of 14 ablation and surface velocity stakes distributed along the central flow line has been established in spring 2008 on the outlet glacier of the A.P. Olsen ice cap and along three transects at elevations of approximately 675, 900 and 1300 m (figure 1 and table. 1) respectively. Each 6 m long stake was assembled from 2 m lengths of aluminium tube. A Kovacs drill was used with success, allowing very fast drilling operations.
6. if time or weather constraints prevent digging a snow pit at every stake, pay particular care when probing to “feel” the snowpack stratigraphy and assess how similar it appears to be to other stake sites where a snow pit has been dug. A snow pit is required in any case if the elevation of the stake site or the snowpack depth are significantly different from the closest snow pit measured. 7.
1. mark in the snow surface the intended outline of the snow pit orienting it so that the side wall where observations will be made is facing away from the sun 2. take a picture documenting the unexcavated site 3. take a note of snow pit position, date, time, pictures number and air temperature 4. dig the snow pit paying attention not to disturb the snow surface along the side where observations will be made 5.
and plugs wiring, and datalogger programming are contained in Appendix B, C and D respectively. Fig. 3 provides an overview of the station marked as AWS 1 in Fig. 1 3 6 5 4 2 1 7 8 9 10 11 Fig. 3 – Overview of the AWS and the sensors. 1. Air temperature and humidity (aspirated); 2. SW↓, LW↓, SW↑, LW↑ radiation; 3. Wind speed and direction; 4. Snow surface level; 5. two-axes tilt meter; 6. Iridium satellite antenna; 7. GPS receiver and Iridium modem; 8. Air pressure; 9.
Required equipment: The following table 2 (prepared by Søren Nielesn, GEUS) describes various recommended set of tools to setup or carry out maintenance on a GlacioBasis AWS. Some of the items may only be required when testing the instruments at Zackenberg Station and may be left behind when leaving for the glacier. Svensknøgler Svensknøgle Løse skraldefastnøgler MINIMUM 8" el.
taken when the available equipment or other technical circumstances prevent operations in DGPS mode. The detailed procedure for taking DGPS measurements vary with the specific receiver model used, see the instrument documentation for details. Whatever the instrument used, the model, configuration and setup must be recorded and special care must be paid in properly configurating the antenna parameters and in positioning the antenna on the stake. Unobstructed view of the sky must be ensured.
1389 1337 P. O L S E N 1416 L A N D Fig. 2 – GPR tracks followed during the 2008 snow depth survey. Reasonably accurate positioning of the GPS traces is important, even though D-GPS accuracy is not a strict requirement for snow depth monitoring.
Contacts This is the updated list of contact details including phone numbers and addresses for use during the fieldwork season fieldwork. It is mainly for the personnel in the field, but also contains some contacts useful during the trip to Greenland and for arranging unanticipated shipments. name Michele Citterio email phone mobile (*) mcit@geus.dk +45 38142113 Andreas Ahlstrøm Søren Nielsen apa@geus.dk +45 38142794 sn@geus.dk +45 38142321 Marianne Vestergaard mve@geus.
* several phone numbers are not published here and must be obtained before leaving Denmark. The complete page is available from within the GEUS intranet in this folder: \\Geusnt1\glaciologi\GlacioBasis\GlacioBasis Manual\contacts.doc Boxes and all cargo to be delivered to GEUS shall always be addressed to: GEUS varemodtagelsen, Riegensgade 13, DK-1316 Copenhagen K Alternatively and only upon agreement with the logistic staff at Zackenberg Station, the delivery can be routed through DMU in Roskilde.
Appendix A - AWS establishment and maintenance checklist (Dirk van As, GEUS) 16
AWS maintenance / establishment checklist version 5 Station name Observer(s) Purpose of visit Establishment of new AWS: Go to Metadata after maintenance / establishment Metadata upon arrival Date & time of arrival at station Time difference between logger clock and UTC Adjust logger time to UTC Download data to PC and/or change CF Card (wait for green light) Download logger program to PC Note or photograph current values in fast scan mode Latitude (dd mm.mmm) Longitude (dd mm.
Maintenance step-by-step Change Internal battery? Data and program will be lost! Change vent filter in logger enclosure? Change wires or aluminum of tripod? Change sonic rangers membranes or replace by new sensors Align CNR1 and inclinometer with mast within 0.
Metadata after maintenance / establishment Latitude (dd mm.mmm) Longitude (dd mm.mmm) Photos of tripod, sensors, logger box wiring etc. Mast inclination in CNR1 direction (+ if CNR1 tilting down) Mast incl.
Serial numbers of hardware SR50 Snow (sonic ranger on AWS) Replaced with: SR50 Abl (sonic ranger on stakes) Replaced with: Rotronic assembly (Assembly no. if new assembly) Hygroclip S3 no. replaced with: MP100H no. replaced with: RS12T (Fan) no. replaced with: MKRS (Arm) no.
Appendix B – AWS station design (Michele Citterio, GEUS) 21
Glaciobasis Main AWS Rev. MCIT 17 Mar 2008 NOTE: This is the working setup with program revision 1.3 Design changes or additions from the previous revision (9 Feb 2008) are written like this. See also the list of changes at the end of the document.
G 5V G SW-12 G 12V 12V G C1 COM1 C2 C3 COM2 C4 G C5 COM3 C6 C7 COM4 C8 G RS232 Ground lug Bottom wiring strip open open BLK Setra barometer SW-12V DISTRIB.
MULTIPLEXER (it must be configured as a 4x16 MUX) COMMON Multiplexer side RES CONTROL CLK & GND POWER 12V H ODD L GND H EVEN L GND H 1 L 1 GND H 2 L GND H 3 L GND 2 H 4 L GND H 5 L GND 3 H 6 L GND H 7 L GND 4 H 8 L GND H 9 L GND 5 H 10 L GND H 11 L 6 GND 12 H Device side C1 C2 G SW-12V DISTRIB.
L GND H L GND H 14 L GND H 15 L GND H 16 L GND H 17 L GND H 18 L GND H 19 L GND H 20 L GND H 21 L GND H 22 L GND H 23 L GND H 24 L GND H 25 L GND H 26 L GND H 27 L GND H 28 L 13 7 8 9 10 11 12 13 14 open open RED BLU open open open open GRY YEL open open open shield WHI BLK open open open open BRO GRE open open open open WHI BRO GRY open open open WHI BRO GRY open open open YEL BLU WHI open open open open open open open open CNR1 – Radiometers cable CNR1 – Radiometers cable CNR1 – Radiometers cab
GND H 29 L GND H 30 L GND H 31 L GND H 32 L GND 15 16 open open open open open open open open open open open open open SW-12V DISTRIBUTION PANEL Circuit side Just wire everything together Sensor side SW-12 GRE GRE RED MUX 12V RED RED BRO Device name CR1000 HigroClip n. 1 HigroClip n. 2 Setra barometer Multiplexer CONTROL & POWER SR50 n.1 SR50 n.2 Tilt sensor LED The LED ishould be a low current type not to overload the CR1000 output which are specified to max. 2 mA. Buy RS code n.
CURRENT SENSING SHUNTS (see drawing for the bridge circuit and the meaning of the node letters below) Circuit side 15 Datalogger GND 12 16 4 8 9 10 13 14 6 5 2 1 17 SP(-) BAT(-) TxGND FanGND SP-H SP-L BAT-H BAT-L FAN-H FAN-L TX-H TX-L GND BOLT other side POWER IN G GND LUG Solar panel negative Batteries negative Tx ground return Fan ground return MUX SE5H MUX SE5L MUX SE3H MUX SE3L MUX SE7H MUX SE7L MUX SE9H MUX SE9L GND BOLT Device name CR1000 Solar panels Batteries (the negative terminals are all tied
RS232 GENDER CHANGER AND TX-RX CROSSOVER ADAPTER (numbers refer to a standard 9 pins D-type connector, see drawing) SYN-DC-936 1 2 3 4 5 6 7 8 9 CR1000 RS232 open 3 2 6 5 4 8 7 open 1 9 viewed from the pins side List of design changes 1. The LED is moved from C3 to C5 in order to leave C3 unused. This is required according to the CR1000 documentation because we are using COM2 as a serial port (for the Garmin) 2.
Appendix C – AWS plugs and internal wirings (Søren Nielsen, GEUS) 29
Plug no.
Plug numbers and positions for ZAK MAIN AWS ZAK SECONDARY AWS only uses plug 1 - 6 9 pole 9 pole 11: Thermist. 6: SR50 Abl 7: Incl 8: SR50 Snow 9: CNR Temp 9,5 10: CNR Sig 6,5 Í 4P male 1: Battery Vent 4 3,5 4P male 2: Solar P 6,5 4: Tasp S3 3: WS&D 5 5 Iridium Ant 14: Tasp2 Fan 13: Tasp2 Sig 12: NT1400 5 5 5 5: Tasp Fan-Pt100 5 5 Unmarked plugs are femal 7 pole.
Battery Group A + Logger, Sensors Terminal strip Solar Panel Charge in Grey A Pin 3 Battery Group B + Pin 1 12V A out Brown Pin 2 12V B out Black B Fan, SatCom, GPS A: B: 2 Diodes 1N5450 (RS Comp 348-5460) Diode 80SQ045 (RS Comp 254-0730) Pin G: All negative battery terminals Blue Plug: 4 pole female 32
PLUGS 1-6 ZAK_M + ZAK_S1 Plug no. Pin no. Plug 1 Batterycable Black cable 1,5 mm² Plug 2 Solar panel Blue cable 0,75 mm² Plug 3 WS&D Sensor cable Sensor cable Blue Brown Black Grey Blue Brown 1 2 3 1 2 3 Shield 1 Black 2 White 3 Green 4 Blue 5 Red 6 Plug 4 Drain 1 Red Tasp A Hygroclip + 2 Black Cable comp.
PLUGS 7-14: ZAK_M only Plug no. Plug 7 Inclinometer Pin no. 1 2 9720/1 cable 3 4 5 6 Plug 8 1 SR50 Snow 2 Adress 0 3 4 5 6 Plug 9 1 CNR-1 temp. 2 Sensor cable 3 4 5 6 1 Plug 10 2 CNR-1 signal 3 Sensor cable 4 5 6 7 8 9 1 Plug 11 2 Thermistor 3 string 4 Cable : 5 Farnell 6 123-5597 7 8 9 Plug 12 1 NT1400 2 Sensor cable 3 4 5 6 Plug 13 1 Tasp A Hygroclip + 2 Cable comp.
Internal ZAK_M Internal Barometer Internal Iridium modem Function Ext. Trigger nc GND 12V VOUT Pin 1 Pin 2 Pin 3 Pin 4 Pin 5 RS232 1 2 3 4 5 6 7 8 9 NAL female changed to RS232 male Iridium anten.
Internal ZAK_S1 Function Power 12 V Power ground Remote On/off Port 1 data in Port 1 data out PPS Port 2 data in Port 2 data out Shield Internal Garmin GPS Red Black Yellow Blue White Grey Green Violet Drain Interface 1 Power measurement SE13 Yellow Fan G Plug 5 pin 2 (Black) SE12 Green Solar Panel G Plug 2 pin SE10 White Battery G Plug 1 pin nc H.
Appendix D – Datalogger program (Michele Citterio, GEUS) 37
'CR1000 Program for the GlacioBasis GEUS AWSs rev. 1.4M (23/03/2008). 'Written by Michele Citterio, GEUS, Copenhagen.
Public SkippedScans Public SkippedSlowScans1, SkippedSlowScans2 Public GPSinUse As String * 1 Public TimeSince12Von Public TimeSinceVx3on Public LastSuccessfulMOMSN Public SendRecordBacklog Public SendRecordIT Public TxTimeTaken Public TxSendAttempts Public TxTimeoutTimer Public TxInitilizeAttempt Public SBDIoffset Public SBDIsentence As String * 40 Public SBDIparsed(6) As String *10 Public SBDsession As Boolean Public TxInitializeOk As Boolean Public StrIndex Public TxSendSuccess As Boolean Public SentWith
Public IsSummer As Boolean Public Batt_VoltI Public Batt_VoltF Public Batt_V_Drop Public Low_Batt As Boolean Public Asp_fan_on As Boolean Public Aspirated_meas As Boolean Public PTemp_C Public SR50_SnowHeight(2) Alias SR50_SnowHeight(1)=SnowHeight ''sr50 Alias SR50_SnowHeight(2)=SnowHeightQuality ''sr50 Public SR50_Ablation(2) Alias SR50_Ablation(1)=Ablation ''sr50 Alias SR50_Ablation(2)=AblationQuality Public Ablation_meter ''sr50 Public temptemp,temptemptemp 'Public WaterColumnPressure Public WS_ms ''Youn
Alias Alias Alias Alias Alias Alias Alias Alias Alias Alias Alias ParseStr(8) = NUMSATS ParseStr(9) = HDP ParseStr(10) = ALTDE ParseStr(11) = ALTUNIT ParseStr(12) = GIODAL ParseStr(13) = GEOUNIT ParseStr(14) = AGE ParseStr(15) = DIFFREF ParseStr(16) = ASTERISK ParseStr(17) = CHCKSUM ParseStr(18) = CRLF Units Batt_VoltI=Volts Units Batt_VoltF=Volts Units Batt_V_Drop=Volts Units PTemp_C=Deg C Units WS_ms=meters/second Units WindDir=Degrees Units BP_mmHg=mV (mmHg) 'Units WaterColumnPressure=mV/V Units AS_T=m
Average(1,CNR1_SWin,FP2,False) ' StdDev(1,CNR1_SWin,FP2,False) Average(1,CNR1_SWout,FP2,False) ' StdDev(1,CNR1_SWout,FP2,False) Average(1,CNR1_LWin,FP2,False) ' StdDev(1,CNR1_LWin,FP2,False) Average(1,CNR1_LWout,FP2,False) ' StdDev(1,CNR1_LWout,FP2,False) Average(1,CNR1_Pt100,FP2,False) ' StdDev(1,CNR1_Pt100,FP2,False) Sample(1,SnowHeight,FP2) Sample(1,SnowHeightQuality,FP2) Sample(1,Ablation,FP2) Sample(1,AblationQuality,FP2) Average(1,Ablation_meter,FP2,False) ' StdDev(1,Ablation_meter,FP2,False) Average(
Sample(1,SnowHeight,FP2) Sample(1,SnowHeightQuality,FP2) Sample(1,Ablation,FP2) Sample(1,AblationQuality,FP2) Average(1,Ablation_meter,FP2,False) ' StdDev(1,Ablation_meter,FP2,False) Average(8,TermStringT(),FP2,False)''check if ok this way of passing arrays FieldNames("Thermistor_1,Thermistor_2,Thermistor_3,Thermistor_4,Thermistor_5,Th Average(1,Xtilt,FP2,False) ' StdDev(1,Xtilt,FP2,False) Average(1,Ytilt,FP2,False) ' StdDev(1,Ytilt,FP2,False) Sample(1,TIME,String) Sample(1,LAT,String) Sample(1,LONGI,String
Sample(1,QUAL,String) Sample(1,NUMSATS,String) Sample(1,HDP,String) Average(1,US_Pt100,IEEE4,False) Average(1,US_T,IEEE4,False)'US_ = Rotronics UN-Aspirated Shield Average(1,US_RH,IEEE4,False) Average(1,PTemp_C,IEEE4,False) Maximum(1,Batt_VoltI,IEEE4,False,False) Minimum(1,Batt_VoltF,IEEE4,False,False) Average(4,Currents(),FP2,False) FieldNames("Battery_current_avg,Solar_Panel_current_avg,Fan_current_avg,Tx+GPS_ EndTable DataTable(TableDiagnostics, True, -1) DataInterval(0,DiagnosticsRate,hr,100) CardOut(0,
EndIf Next SerialClose(ComRS232) EndIf EndSub Sub IridiumTx If NOT (SummerTxRate OR WinterTxRate OR (GPSType="N")) Then ExitSub Call TxModuleInit If NOT TxInitializeOk Then SerialClose(ComRS232) ExitSub EndIf SerialOpen(ComRS232,19200,0,0,400) If SerialOut(ComRS232,"AT*S=1"+CHR(13)+CHR(10),"OK"+CHR(13)+CHR(10),1,10) Then If SerialOut(ComRS232,"AT+SBDD0"+CHR(13)+CHR(10),"0",1,10) Then TxMObuffCleare If SerialOut(ComRS232,"AT+SBDWT"+CHR(13)+CHR(10),"READY" + CHR(13) + CHR(10), SerialOutBlock (ComRS232,TxStri
Else TxSendSuccess = False TxSvcAvail = false TxRSSI = 0 EndIf SBDsession = False'\ Loop Until TxSendSuccess' actually, it keeps trying until TxTimeout TxTimeTaken = TxTimeoutTimer Timer(1,sec,3) SerialClose(ComRS232) EndSub '******************************************************************************* ' The main program. On datalogger power-on and after a few seconds needed for th ' lights the LED for 3 seconds to notify that the station is up and running. The ' the only SlowSequence Scan.
' If Batt_VoltI > LoBattThre + LoBattResHyst Then Low_Batt = False 'Initial Datalogger Battery Voltage measurement Batt_Volt (do this before turn ' If Batt_VoltI = -999 Then Battery(Batt_VoltI) PulsePort(5,10000) 'Excites Vx3 to allow the NT1400 to thermally stabilize If (TimeIntoInterval(0,ScanRate,sec) OR FastScanMode) Then ExciteV(Vx3,2500,0) Timer(3,sec,2) TimeSinceVx3on = Timer(3,sec,4) Else Timer(3,sec,3)'we won't need to also switch Vx3 off explicitly: it is done EndIf 'deals with the fastscan state
temptemp=temptemp+1 PortSet(6,True) TxModuleGPSPowerOn = True SerialOpen(ComRS232,19200,0,0,100) SerialFlush(ComRS232) Call TxModuleInit If TxInitializeOk Then SerialOut(ComRS232,"AT*S=0"+CHR(13)+CHR(10),"",1,10) SerialOut(ComRS232,"AT+PP=1"+CHR(13)+CHR(10),"",1,10)'should be SerialOut(ComRS232,"AT+PNAV=1"+CHR(13)+CHR(10),"",1,10)'should b SerialClose(ComRS232) EndIf EndIf If NOT(GarminInitOk OR TxInitializeOk) Then TxPowerDueForSwOff = True EndIf EndIf Else'else it is winter If (TimeIntoInterval(WinterGPSr
Timer(2,sec,2) TimeSince12Von = Timer(2,sec,4) 'Do the measurements '\'SerialOpen(com2,4800,0,0,150)'due to a possible bug in the CR1000 OS, i 'Wiring Panel Temperature measurement PTemp_C: PanelTemp(PTemp_C,_50Hz) ExciteV(Vx3,2500,0)'This MUST be right after any bridge, P107 and Pa 'get a feel of the season...
Delay(0,20,msec)' allows for settling time ... needed?? VoltDiff (CNR1_SWin,1,mV25,7,True,0,_50Hz,1,0)' may go out of range with ver PulsePort(2,5000) Delay(0,20,msec)' allows for settling time ... needed?? VoltDiff (CNR1_LWin,1,mV7_5,7,True,0,_50Hz,1,0)' may go out of range with ve PulsePort(2,5000) Delay(0,20,msec)' allows for settling time ... needed?? VoltDiff (CNR1_SWout,1,mV25,7,True,0,_50Hz,1,0) PulsePort(2,5000) Delay(0,20,msec)' allows for settling time ...
'resets the MUX (so that even if 12V switched is powered, the MUX itself shoul PortSet(1,false) 'do now whatever needs the 12V switched supply to be ON and requires warm-up, bu 'GPS work for Garmin (G) or NAL (N) units, or both! PreParseStr(7) = 0 PreParseStr(9) = 0 'The Garmin part finds its com2 already open and the prog closes it afterwar 'around the CR1000 bug causing higher power draw if we open/close com2 here If GarminInitOk AND TxModuleGPSpowerOn AND (GPSinUse = "G") Then GGPSFLAG = true GPGGAsente
TxInitializeOk = False EndIf If NOT (FastScanMode OR LastFastScan) Then SerialOut(ComRS232,"AT+PP=0"+CHR(13)+CHR(10),"",1,10) EndIf SerialClose(ComRS232) NGPSFLAG = false TxPowerDueForSwOff = True ElseIf (FastScanMode OR LastFastScan) Call TxModuleInit If TxInitializeOk Then SerialOut(ComRS232,"AT*S=0"+CHR(13)+CHR(10),"",1,10) SerialOut(ComRS232,"AT+PP=1"+CHR(13)+CHR(10),"",1,10)'should be sent as SerialOut(ComRS232,"AT+PNAV=1"+CHR(13)+CHR(10),"",1,10)'should be sent a SerialClose(ComRS232) EndIf If NOT (Be
TxBufferString=Left(TxBufferString, Len(TxBufferString)-2)+","+Mid(Tab EndIf CallTable(TxBufferTable) If SendRecordBacklog < SummerTableBuffer Then SendRecordBacklog = SendRe If TimeIntoInterval(0,DiagnosticsRate,hr) AND SendRecordBacklog < Summ TxBufferString="" GetRecord(TxBufferString,TableDiagnostics,1) CallTable(TxBufferTable) SendRecordBacklog = SendRecordBacklog + 1 EndIf For SendRecordIT = SendRecordBacklog To 1 Step -1 TxString="" MpFLAG=true'remove!! Delay(0,2,sec)'remove!! 'Call AppendInstantMeas
For StringStripperIT=StripStringStart To Len(TxStringUnstripped)-5'5 i StringChar=Mid(TxStringUnstripped, StringStripperIT, 1) If StringChar<>CHR(34) Then TxString=TxString+StringChar Next StringStripperIT TxString=TxString+",!M"'*************in summer too 'TxString=Right(TxString,Len(TxString)-InStr(1,TxString,CHR(44),4)+0) Call IridiumTx MPFLAG=false'remove!! If NOT TxSendSuccess Then ExitFor Next If SendRecordIT = 0 Then ResetTable(TxBufferTable) SendRecordBacklog = SendRecordIT TxPowerDueForSwOff = True
Appendix E – Telemetry data retrieval program (Michele Citterio, GEUS) 55
#getdata 1.60f, written by Michele Citterio (mcit@geus.dk) at GEUS, Copenhagen #coding=cp850 # #install and use: install pyhton 2.5.2 and pywin32 212, copy this file into an #empty folder and run it from there. Four files and a folder can be created #for each IMEI number found when retrievig the emails from the Exchange server: #1) IMEI#.txt readable comma-separated ascii file - AWS observations #2) IMEI#-D.txt readable comma-separated ascii file - AWS diagnostics #3) IMEI#-F.
# known issues: the stats about the processed messages and some screen output # while running are wrong (but the data files generated are ok). # Also, when user aborts a file write error, the line is still counted as added # Cause is known and harmless, will be fixed in 1.60 final.
#there's about 10^-18 chances of an error after 10^10 messages received...
#what follows should be turned into methods of a future FormatSpecTable object FormatSpecAreOK = True for FormatSpecEntry in FormatSpec.iteritems(): #FormatSpecEntry[1][0] if FormatSpecEntry[1][0] != len(FormatSpecEntry[1][1]): print 'WARNING: bad format definition for message type %i (%s):\n format unconsistency (%i values declared but %i specified)' %(FormatSpecEntry[0], FormatSpecEntry[1][2],FormatSpecEntry[1][0],len(FormatSpecEntry[1][1])) FormatSpecAreOK = False#sys.
SeenMessageHashes = [] SeenMessageHashesPreviousRuns = SeenMessageHashes[:]#TODO learn better why this is actually required! AppendedSummer = 0 AppendedWinter = 0 AppendedWithInstant = 0 AppendedDiagnostics = 0 ApendedMalformed = 0 ProcessedInfostoreName = '' ProcessedFolderName = '' try: cdo = Dispatch("MAPI.session") DeafultLoginName = win32api.
InfoStore_Prompt = 'Enter the desired infostore [1, %i]: ' %InfoStoresCount while True: InfoStore = raw_input(InfoStore_Prompt) if not len(InfoStore): break try: InfoStore = int(InfoStore) if InfoStore < 0: raise ValueError if InfoStore > InfoStoresCount: raise ValueError break except ValueError: print "WARNING: not in the range [1, %i]" %InfoStoresCount if not InfoStore: InfoStoreNum = DefaultInfoStoreNum else: InfoStoreNum = InfoStore - 1#needs to be 0-based to be used as an index try: ProcessedInfostoreN
try: Folder = int(Folder) if Folder < 0: raise ValueError if Folder > FoldersCount: raise ValueError break except ValueError: print "WARNING: not in the range [1, %i]" %FoldersCount if not Folder: FolderNum = DefaultFolderNum else: FolderNum = Folder - 1#needs to be 0-based to be used as an index try: ProcessedFolderName = Folders[FolderNum 0].Name.encode(UnicodeEncoding) print 'Looking for messages in %s...' %ProcessedFolderName, Messages = Folders[FolderNum - 0].
MOMSN = AttachmentName[AttachmentName.index('_')+1:AttachmentName.index('.')] ThisIMEIDir = BaseDir + '\\' + IMEI + '\\'#TODO make it more portable using the path module if not glob.glob(ThisIMEIDir): #this IMEI has already been seen os.mkdir(BaseDir + '\\' + IMEI.encode(UnicodeEncoding))#TODO make it more portable using the path module Attachment.
ValueBytes.append(ord(BinaryMessage[BytePointer + offset])) BytePointer = BytePointer + ValueBytesNum Value = GFP2toDEC(ValueBytes) if Value == 8191: DataLine = DataLine + "NAN" elif Value == 8190: DataLine = DataLine + "INF" elif Value == -8190: DataLine = DataLine + "-INF" else: DataLine = DataLine + str(Value) #print ValueBytes, Value except IndexError: DataLine = DataLine + '?' if MessageFormat[1][ValueNum] == 'l': try: for offset in range(0,ValueBytesNum): ValueBytes.
for offset in range(0,ValueBytesNum): ValueBytes.append(ord(BinaryMessage[BytePointer + offset])) BytePointer = BytePointer + ValueBytesNum Value = GLI4toDEC(ValueBytes)/100.0 DataLine = DataLine + str(Value) #print ValueBytes, Value except IndexError: DataLine = DataLine + '?' elif MessageFormat[1][ValueNum] == 'n': try: for offset in range(0,ValueBytesNum): ValueBytes.append(ord(BinaryMessage[BytePointer + offset])) BytePointer = BytePointer + ValueBytesNum Value = GLI4toDEC(ValueBytes)/100000.
elif IsObservations and not IsSummer: print '(ascii) generic winter observations message', else: print 'unrecognized message format', if IsWithInstant: print '(+ instant.)' else: print '' IsMalformed = IsTooLong or IsTooShort if IsMalformed: FoundMalformed += 1 print " WARNING - Message is malformed: any missing value will be replaced by '?'" DataLineHash = HashFunc(IMEI + MOMSN + DataLine).
while True: try: with open(ObservationFilePath, 'a') as OutFile: OutFile.writelines(DataLine+'\n') if not ObservationFilePath in ModifiedFiles: ModifiedFiles.append(ObservationFilePath) break except IOError: while True: AbortOrRetry = raw_input('ERROR while opening %s: Abort or Retry? [A/R]' %ObservationFilePath) if AbortOrRetry.upper() in ('A', 'R'): break if AbortOrRetry.
#DataLine = '' Message = Messages.GetNext()# ref. http://mail.python.org/pipermail/pythonlist/2004-July/270944.html try: with open(BaseDir + '\\msgshash.dat', 'wb') as msgshash: cPickle.dump(SeenMessageHashes, msgshash) except IOError: print "Warning - Couldn't save the hashes of the new messages" except: print 'ERROR: an unhandled exception occurred, terminating the program.' raise finally: if 'LoggedIn' in locals(): try: print '\nlogging off from the Exchange server...', cdo.
print '(using setting: append to IMEI#.txt)' print """ Messages shorter than expected have missing values replaced with ? * Any massage longer than expected is decoded up to the expected length * All original files can be found in the subfolder of the relevant IMEI * Set bool FilterMalformed to control where these data get appended""" DoSort = '' #print '\n' while ModifiedFiles:#so that it only asks if any file has actually been modified DoSort = raw_input('New lines were appended.
