src

Go monorepo.
git clone git://code.dwrz.net/src
Log | Files | Refs

Doc.txt (2598B)

      1 The upload process converts count files into reports, and
      2 uploads reports. There will be only one report, named YYYY-MM-DD.json,
      3 for a given day.
      4 
      5 First phase. Look at the localdir (os.UserConfigdir()/go/telemetry/local)
      6 and find all .count and .json files. Find the count files that are no
      7 longer active by looking at their metadata.
      8 
      9 Second phase. Group the inactive count files by their expiry date, and
     10 for each date generate the local report and the upload report. (The upload
     11 report only contains the counters in the upload configuration.) The upload
     12 report is saved in the local directory with a name like YYYY-MM-DD.json, if
     13 there is no file already existing with that name.
     14 If the local report is different, it is saved in the local directory
     15 with a name like local.YYYY-MM-DD.json. The new upload report is
     16 added to the list of .json files from the first phase. At this point
     17 the count files are no longer needed and can be deleted.
     18 
     19 Third phase. Look at the .json files in the list from the first phase.
     20 If the name starts with local, skip it. If there is a file with the
     21 identical name in the upload directory, remove the one in the local directory.
     22 Otherwise try to upload the one in the local directory,
     23 If the upload succeeds, move the file to the uploaded directory.
     24 
     25 
     26 There are various error conditions.
     27 1. Several processes could look at localdir and see work to do.
     28 1A. They could see different sets of expired count files for some day.
     29     This could happen if another process is removing count files. In this
     30     case there is already a YYYY-MM-DD.json file either in localdir
     31     or updatedir, so the process seeing fewer count files will not generate
     32     a report.
     33 1B. They could see the same count files, and no report in either directory.
     34     They will both generate (in memory) reports and check to see if there
     35     is a YYYY-MM-DD.json file in either directory. They could both then
     36     write two files with the same name, but different X values, but
     37     otherwise the same contents. The X values are very close to the front
     38     of the file. Assuming reasonable file system semantics one version of
     39     the file will be written. To minimize this, just before writing reports
     40     the code checks again to see if they exist.
     41 1C. Once there is an existing well-formed file YYYY-MM-DD.json in localdir
     42     eventually the upload will succeed, and the file will be moved to updatedir.
     43     It is possible that other processes will not see the file in updatedir and
     44     upload it again and also move it to uploaddir. This is harmless as all
     45     the uploaded files are identical.