| 2018-12-13
      
      ยง | 
    
  | 19:53 | <hoo> | Ran scap pull on snapshot1005 to undo live changes done for dump performance testing | [production] | 
            
  | 19:49 | <dcausse> | creating 300 wiki indices in elastic-omega@eqiad | [production] | 
            
  | 19:46 | <dcausse> | SF Morning SWAT done | [production] | 
            
  | 19:44 | <dcausse@deploy1001> | Synchronized wmf-config/CirrusSearch-production.php: T210381: [cirrus] fix cluster settings for temp clusters psi&omega (duration: 00m 44s) | [production] | 
            
  | 19:40 | <moritzm> | removed labvirt1014 from debmonitor DB, has been renamed to cloudvirt1014 | [production] | 
            
  | 19:39 | <volans> | imported python-elasticsearch_5.4.0-1~deb9u1 into apt.w.o stretch-wikimedia component/spicerack - T205884 | [production] | 
            
  | 19:35 | <dcausse@deploy1001> | Synchronized php-1.33.0-wmf.8/extensions/TwoColConflict/: T210501: Add missing code to not loose edits on the other side (duration: 00m 45s) | [production] | 
            
  | 19:29 | <bblack> | authdns2001: upgrading gdnsd to 2.99.9944-beta | [production] | 
            
  | 19:27 | <bblack> | multatuli: upgrading gdnsd to 2.99.9944-beta | [production] | 
            
  | 19:22 | <dcausse@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: T211234: Enable Block notice stats on top blocking wikis (duration: 00m 45s) | [production] | 
            
  | 19:19 | <bblack> | authdns1001: upgrading gdnsd to 2.99.9944-beta | [production] | 
            
  | 18:52 | <arlolra> | Updated Parsoid to 4242ad0 (T204622, T211738) | [production] | 
            
  | 18:40 | <arlolra@deploy1001> | Finished deploy [parsoid/deploy@e27574c]: Updating Parsoid to 4242ad0 (duration: 09m 17s) | [production] | 
            
  | 18:31 | <arlolra@deploy1001> | Started deploy [parsoid/deploy@e27574c]: Updating Parsoid to 4242ad0 | [production] | 
            
  | 18:30 | <mholloway-shell@deploy1001> | Finished deploy [mobileapps/deploy@93443fe]: Refine MW API queries (duration: 03m 41s) | [production] | 
            
  | 18:26 | <mholloway-shell@deploy1001> | Started deploy [mobileapps/deploy@93443fe]: Refine MW API queries | [production] | 
            
  | 18:24 | <jforrester@deploy1001> | Synchronized wmf-config/Wikibase.php: T204748 [Beta only] Use newly-fixed config for Wikibase->Commons federation (duration: 00m 44s) | [production] | 
            
  | 18:22 | <jforrester@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: T204748 Rename repo-only Wikibase config for clarity [no-op] (duration: 00m 45s) | [production] | 
            
  | 18:11 | <bblack> | restart pybal on lvs1006 for config updates | [production] | 
            
  | 18:07 | <ladsgroup@deploy1001> | Synchronized php-1.33.0-wmf.8/extensions/FlaggedRevs/frontend/specialpages/reports/ProblemChanges_body.php: Use the right index for change_tag (T211896) (duration: 00m 46s) | [production] | 
            
  | 17:45 | <ladsgroup@deploy1001> | Finished deploy [ores/deploy@1a3de73]: T211267 (duration: 13m 53s) | [production] | 
            
  | 17:41 | <akosiaris> | reapply the zotero calico policy to allow LVS endpoints | [production] | 
            
  | 17:35 | <dcausse> | elastic@eqiad created cirrus metastore on psi&omega | [production] | 
            
  | 17:32 | <ladsgroup@deploy1001> | Started deploy [ores/deploy@1a3de73]: T211267 | [production] | 
            
  | 17:31 | <arturo> | T168967 added shiny-server .deb to stretch-wikimedia | [production] | 
            
  | 17:20 | <godog> | run puppet and bounce pybal on lvs in eqiad to apply https://gerrit.wikimedia.org/r/c/operations/puppet/+/479184 | [production] | 
            
  | 17:18 | <akosiaris> | T205919 create namespace for blubberoid on eqiad/codfw/staging clusters | [production] | 
            
  | 17:00 | <anomie> | Set comment migration to new on group 1 (T166733) | [production] | 
            
  | 16:33 | <mutante> | icinga1001 - started service again, enabeld puppet | [production] | 
            
  | 16:27 | <mutante> | icinga1001 - disable puppet, stopped icinga, for cable replacement | [production] | 
            
  | 16:18 | <anomie> | Deployed fix for T210937: API: Use parenthesized join in ApiQueryBase::showHiddenUsersAddBlockInfo | [production] | 
            
  | 15:44 | <moritzm> | installing openssl 1.1 security updates on Hadoop workers | [production] | 
            
  | 15:33 | <moritzm> | rebooting pybal-test hosts to pick up SSBD-enabled qemu | [production] | 
            
  | 15:18 | <anomie@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: Setting actor migration to write-both/read-old on all wikis (T188327) (duration: 00m 45s) | [production] | 
            
  | 15:12 | <anomie@deploy1001> | Synchronized php-1.33.0-wmf.8/includes/api/ApiPageSet.php: Backport fix for T211804: ApiPageSet::initFromPageIds: Default $filterIds to true (duration: 00m 46s) | [production] | 
            
  | 15:03 | <akosiaris> | ores2* deploy https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/479206/ | [production] | 
            
  | 14:55 | <moritzm> | installing openssl 1.1 security updates on mw canaries (along with nginx restart/upgrade) | [production] | 
            
  | 14:46 | <cdanis> | updating grafana/stretch-wikimedia to 5.4.2: reprepro --restrict grafana update stretch-wikimedia | [production] | 
            
  | 14:40 | <akosiaris> | disable puppet on ores1* and ores2* machines to deploy https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/479206/ | [production] | 
            
  | 14:36 | <ladsgroup@deploy1001> | Finished deploy [ores/deploy@a9d5e95]: noop (duration: 14m 59s) | [production] | 
            
  | 14:23 | <ladsgroup@deploy1001> | Started deploy [ores/deploy@a9d5e95]: noop | [production] | 
            
  | 14:21 | <zfilipin@deploy1001> | rebuilt and synchronized wikiversions files: all wikis to 1.33.0-wmf.8 | [production] | 
            
  | 13:22 | <dcausse> | creating 300+ wiki indices on elastic-omega@codfw | [production] | 
            
  | 13:15 | <godog> | stop restbase and cassandra on restbase200[1-6] - T211070 | [production] | 
            
  | 12:59 | <elukey> | superset on analytics-tool1003 upgraded to 0.28.1 | [production] | 
            
  | 12:41 | <arturo> | icinga downtime (30 mins) cloudcontrol1003, cloudnet1003 and cloudnet1004 for package upgrades | [production] | 
            
  | 12:38 | <zeljkof> | EU SWAT finished | [production] | 
            
  | 12:35 | <zfilipin@deploy1001> | Synchronized static/images/project-logos: SWAT: [[gerrit:479411|Update maiwiki logo (T211845)]] (duration: 00m 52s) | [production] | 
            
  | 12:35 | <oblivian@deploy1001> | scap-helm zotero finished | [production] | 
            
  | 12:35 | <oblivian@deploy1001> | scap-helm zotero cluster codfw completed | [production] |