| 2019-11-15
      
      § | 
    
  | 22:14 | <jeh@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 22:12 | <jeh@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 21:54 | <jeh@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 21:52 | <jeh@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 21:31 | <jeh@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 21:29 | <jeh@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 21:21 | <_joe_> | disabling proxying to ws on phabricator1003 | [production] | 
            
  | 20:04 | <XioNoX> | push pfw policies to pfw3-eqiad - T238368 | [production] | 
            
  | 20:02 | <XioNoX> | push pfw policies to pfw3-codfw - T238368 | [production] | 
            
  | 19:07 | <XioNoX> | remove vlan 1 trunking between msw1-codfw and mr1-codfw, will cause a quick connectivity issue - T228112 | [production] | 
            
  | 18:07 | <XioNoX> | homer push on management switches | [production] | 
            
  | 17:30 | <mutante> | phabricator - -started phd service | [production] | 
            
  | 17:11 | <XioNoX> | homer push to management routers (https://gerrit.wikimedia.org/r/550576) | [production] | 
            
  | 16:43 | <hashar> | Restored zuul-merger / CI for operations/puppet.git | [production] | 
            
  | 16:29 | <hashar> | CI slowed down due to a huge spike of internal jobs.  Being flushed as of now # T140297 | [production] | 
            
  | 16:25 | <bblack> | repool cp2001 | [production] | 
            
  | 16:08 | <bblack> | depool cp2001 for experiments | [production] | 
            
  | 16:02 | <moritzm> | rebooting rpki1001 to rectify microcode loading | [production] | 
            
  | 16:00 | <jmm@cumin2001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 16:00 | <jmm@cumin2001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 15:51 | <ejegg> | updated Fundraising CiviCRM from ae9b3819cd to c05c302e54 | [production] | 
            
  | 15:36 | <ejegg> | reduced batch size of CiviCRM contact deduplication jobs | [production] | 
            
  | 15:11 | <ema> | pool cp3064 with ATS backend T227432 | [production] | 
            
  | 15:07 | <ema> | reboot cp3064 after reimage | [production] | 
            
  | 14:51 | <ema@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 14:49 | <ema@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 14:25 | <ema> | depool cp3064 and reimage as text_ats T227432 | [production] | 
            
  | 14:17 | <godog> | SIGHUP prometheus@ops on prometheus1004 | [production] | 
            
  | 14:13 | <bblack> | lvs1013 - pybal restart for new config | [production] | 
            
  | 14:13 | <bblack> | lvs2001 - pybal restart for new config | [production] | 
            
  | 14:13 | <bblack> | lvs5001 - pybal restart for new config | [production] | 
            
  | 14:13 | <bblack> | lvs4005 - pybal restart for new config | [production] | 
            
  | 14:12 | <bblack> | lvs3005 - pybal restart for new config | [production] | 
            
  | 14:11 | <bblack> | lvs5003 - pybal restart for new config | [production] | 
            
  | 14:11 | <bblack> | lvs4007 - pybal restart for new config | [production] | 
            
  | 14:11 | <bblack> | lvs3007 - pybal restart for new config | [production] | 
            
  | 14:10 | <bblack> | lvs2004 - pybal restart for new config | [production] | 
            
  | 14:09 | <bblack> | lvs1016 - pybal restart for new config | [production] | 
            
  | 13:28 | <ariel@deploy1001> | Finished deploy [dumps/dumps@61090ee]: configuration setting to produce empty abstracts (duration: 00m 03s) | [production] | 
            
  | 13:28 | <ariel@deploy1001> | Started deploy [dumps/dumps@61090ee]: configuration setting to produce empty abstracts | [production] | 
            
  | 13:06 | <ariel@deploy1001> | Finished deploy [dumps/dumps@61090ee]: configuration setting to produce empty abstracts (expecting failure) (duration: 00m 04s) | [production] | 
            
  | 13:06 | <ariel@deploy1001> | Started deploy [dumps/dumps@61090ee]: configuration setting to produce empty abstracts (expecting failure) | [production] | 
            
  | 11:43 | <ariel@deploy1001> | Finished deploy [dumps/dumps@61090ee]: configuration setting to produce empty abstracts (duration: 00m 09s) | [production] | 
            
  | 11:43 | <ariel@deploy1001> | Started deploy [dumps/dumps@61090ee]: configuration setting to produce empty abstracts | [production] | 
            
  | 11:27 | <moritzm> | reboott ganeti4001-4003 to rectify microcode application | [production] | 
            
  | 11:26 | <jmm@cumin2001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] |