| 2018-12-06
      
      ยง | 
    
  | 23:47 | <ppchelko@deploy1001> | Finished deploy [recommendation-api/deploy@299b268]: Add 'morelike' article recommendations API T201192 (duration: 02m 06s) | [production] | 
            
  | 23:47 | <XioNoX> | troubleshoot bird bfd on dns2001/cr1-codfw | [production] | 
            
  | 23:45 | <ppchelko@deploy1001> | Started deploy [recommendation-api/deploy@299b268]: Add 'morelike' article recommendations API T201192 | [production] | 
            
  | 23:21 | <ppchelko@deploy1001> | Finished deploy [restbase/deploy@be8f0c0]: Add 'morelike' recommendation public API specification T201192 (duration: 22m 46s) | [production] | 
            
  | 22:58 | <ppchelko@deploy1001> | Started deploy [restbase/deploy@be8f0c0]: Add 'morelike' recommendation public API specification T201192 | [production] | 
            
  | 22:12 | <urandom> | decommissioning cassandra-b, restbase2001 -- T210843 | [production] | 
            
  | 21:39 | <gtirloni> | reimaging cloudvirt1019 with jessie T196507 | [production] | 
            
  | 21:33 | <ppchelko@deploy1001> | Finished deploy [changeprop/deploy@f675fcc]: Added performer to the revision-scores event (duration: 01m 15s) | [production] | 
            
  | 21:32 | <ppchelko@deploy1001> | Started deploy [changeprop/deploy@f675fcc]: Added performer to the revision-scores event | [production] | 
            
  | 21:07 | <smalyshev@deploy1001> | Finished deploy [wdqs/wdqs@cbe4551]: Install new Updater with INSERT DATA (duration: 09m 18s) | [production] | 
            
  | 21:00 | <XioNoX> | remove 2 esams avoid path + 4 prefered/selected transits - T194542 | [production] | 
            
  | 20:58 | <smalyshev@deploy1001> | Started deploy [wdqs/wdqs@cbe4551]: Install new Updater with INSERT DATA | [production] | 
            
  | 20:51 | <XioNoX> | remove 2 eqiad avoid path - T194542 | [production] | 
            
  | 20:48 | <gtirloni> | reimaging cloudvirt1019 with stretch T196507 | [production] | 
            
  | 20:45 | <XioNoX> | remove codfw/eqdfw avoid path - T194542 | [production] | 
            
  | 19:32 | <gehel> | shutting down elasticsearch on elastic2001-2024 (third time is a charm) - T211023 | [production] | 
            
  | 18:21 | <mholloway-shell@deploy1001> | Finished deploy [mobileapps/deploy@1dba3cd]: Internally promisify page processing steps (T202642) (duration: 03m 54s) | [production] | 
            
  | 18:17 | <mholloway-shell@deploy1001> | Started deploy [mobileapps/deploy@1dba3cd]: Internally promisify page processing steps (T202642) | [production] | 
            
  | 17:31 | <moritzm> | installing nodejs updates on proton* | [production] | 
            
  | 17:11 | <moritzm> | uploaded nodejs 6.11~dfsg-1+wmf5 for jessie-wikimedia (the upstream patch for CVE-2018-12122 had a regression, this update fixes it) | [production] | 
            
  | 16:27 | <urandom> | decommissioning cassandra-a, restbase2001 -- T210843 | [production] | 
            
  | 16:17 | <gehel> | shutting down elasticsearch on elastic2001-2024 (second try) - T211023 | [production] | 
            
  | 15:51 | <moritzm> | upgrading spamassassin on mx1001/fermium | [production] | 
            
  | 15:45 | <fsero@deploy1001> | scap-helm zotero finished | [production] | 
            
  | 15:45 | <fsero@deploy1001> | scap-helm zotero cluster codfw completed | [production] | 
            
  | 15:45 | <fsero@deploy1001> | scap-helm zotero upgrade production -f ../zotero-values-codfw.yaml stable/zotero [namespace: zotero, clusters: codfw] | [production] | 
            
  | 15:42 | <fsero> | modifying zotero deploy CLUSTER=codfw scap-helm zotero upgrade production -f zotero-values-codfw.yaml stable/zotero - T211322 | [production] | 
            
  | 15:42 | <_joe_> | disabling puppet fleet-wide for a change in the role() function | [production] | 
            
  | 15:38 | <moritzm> | uploaded nodejs 6.11~dfsg-1+wmf5 for stretch-wikimedia (the upstream patch for CVE-2018-12122 had a regression, this update fixes it) | [production] | 
            
  | 15:08 | <gehel> | restartign new elasticsearch masters on codfw - T211023 | [production] | 
            
  | 15:05 | <gehel> | upgrade nginx on wdqs servers | [production] | 
            
  | 14:59 | <elukey@deploy1001> | Finished deploy [analytics/turnilo/deploy@6bd6e2f]: upgrade deps to nodejs 10 (duration: 00m 09s) | [production] | 
            
  | 14:59 | <elukey@deploy1001> | Started deploy [analytics/turnilo/deploy@6bd6e2f]: upgrade deps to nodejs 10 | [production] | 
            
  | 14:46 | <moritzm> | uploaded nodejs 10.4.0~dfsg-1+wmf2 to apt.wikimedia.org/component/node10 (backports of recent security fixes) | [production] | 
            
  | 14:16 | <moritzm> | installing nginx security updates on mw in eqiad | [production] | 
            
  | 13:46 | <moritzm> | upgrading spamassassin on mx2001 | [production] | 
            
  | 12:56 | <gehel> | depooling and shutting down elasticsearch on elastic2001-2024 - T211023 | [production] | 
            
  | 12:55 | <moritzm> | installing nginx updates on mw in codfw | [production] | 
            
  | 11:59 | <moritzm> | installing nginx updates on mw canaries | [production] | 
            
  | 10:59 | <banyek@deploy1001> | Synchronized wmf-config/db-eqiad.php: T85757: repool db1096:3315 (duration: 00m 47s) | [production] | 
            
  | 10:56 | <volans> | disable event handler on Icinga for ms-be2047 MD Raid and MegaRAID checks, it's spamming Phabricator - T209921 | [production] | 
            
  | 10:56 | <banyek> | repooling db1096 for schema change - T85757 | [production] | 
            
  | 10:36 | <banyek@deploy1001> | Synchronized wmf-config/db-eqiad.php: T85757: depool db1096:3315 (duration: 00m 49s) | [production] | 
            
  | 10:29 | <banyek> | depooling db1096 for schema change - T85757 | [production] | 
            
  | 09:56 | <dcausse> | elastic@codfw cleanup: deleting wikidatawiki_content_1537469318 index (failed reindex probably) | [production] | 
            
  | 01:03 | <tstarling@deploy1001> | Synchronized w/fatal-error.php: (no justification provided) (duration: 00m 46s) | [production] | 
            
  | 00:55 | <tstarling@deploy1001> | Synchronized w/fatal-error.php: (no justification provided) (duration: 00m 47s) | [production] | 
            
  | 00:42 | <tstarling@deploy1001> | Synchronized w/fatal-error.php: (no justification provided) (duration: 00m 46s) | [production] | 
            
  | 00:40 | <tstarling@deploy1001> | Synchronized private/FatalErrorSettings.php: (no justification provided) (duration: 00m 46s) | [production] | 
            
  | 00:38 | <tstarling@deploy1001> | Synchronized private/FatalErrorSettings.php: (no justification provided) (duration: 00m 46s) | [production] |