| 2018-12-05
      
      ยง | 
    
  | 16:15 | <akosiaris@deploy1001> | scap-helm zotero cluster eqiad completed | [production] | 
            
  | 16:15 | <akosiaris@deploy1001> | scap-helm zotero upgrade production -f zotero-values-eqiad.yaml stable/zotero [namespace: zotero, clusters: eqiad] | [production] | 
            
  | 16:08 | <fsero> | redeploying zotero on eqiad | [production] | 
            
  | 16:02 | <akosiaris@deploy1001> | scap-helm zotero finished | [production] | 
            
  | 16:02 | <akosiaris@deploy1001> | scap-helm zotero cluster codfw completed | [production] | 
            
  | 16:02 | <akosiaris@deploy1001> | scap-helm zotero cluster eqiad completed | [production] | 
            
  | 16:02 | <akosiaris@deploy1001> | scap-helm zotero upgrade production --set resources.replicas=16 stable/zotero [namespace: zotero, clusters: eqiad,codfw] | [production] | 
            
  | 16:02 | <akosiaris@deploy1001> | scap-helm zotero upgrade production --set resources.replicas=16 [namespace: zotero, clusters: eqiad,codfw] | [production] | 
            
  | 15:34 | <thcipriani> | restarting ci jenkins for update | [production] | 
            
  | 15:33 | <akosiaris> | add back pods/portforward right to kubernetes deploy user. T211040 | [production] | 
            
  | 15:07 | <anomie> | Running cleanupUsersWithNoId.php on potentially missed s3 and s7 wikis for T181731 | [production] | 
            
  | 15:03 | <fsero> | repooling citoid mathoid eqiad | [production] | 
            
  | 15:03 | <godog> | bootstrap cassandra-c, restbase2017 - T210843 | [production] | 
            
  | 14:56 | <banyek> | executing schema change on s5 codfw master replication lag could be expected - T85757 | [production] | 
            
  | 14:54 | <fsero> | upgrading k8s on eqiad to 1.10.11 | [production] | 
            
  | 14:54 | <elukey> | restart HDFS namenode and Yarn resource manager on an-master100[1,2] to update rack topology config - T209929 | [production] | 
            
  | 14:51 | <fsero> | depool mathoid/eqiad: pooled changed True => False | [production] | 
            
  | 14:51 | <fsero> | depool citoid/eqiad: pooled changed True => False | [production] | 
            
  | 14:43 | <anomie> | Running cleanupUsersWithNoId.php on metawiki for T181731 / T210985 | [production] | 
            
  | 14:09 | <hoo@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: Enable arbitrary item/property access for all wiktionaries (T175273) (duration: 00m 47s) | [production] | 
            
  | 13:53 | <akosiaris> | repool citoid/mathoid codfw | [production] | 
            
  | 12:59 | <onimisionipe> | banning elastic2001-elastic2024 from codfw production, psi and omega clusters | [production] | 
            
  | 12:53 | <jijiki> | uploaded python-thumbor-community-core_0.4.0-1+deb9u1 to stretch-wikimedia | [production] | 
            
  | 12:47 | <dcausse> | EU SWAT done | [production] | 
            
  | 12:45 | <oblivian@puppetmaster1001> | conftool action : set/pooled=false; selector: dnsdisc=(cit|math)oid,name=codfw | [production] | 
            
  | 12:44 | <dcausse@deploy1001> | Synchronized wmf-config/CirrusSearch-production.php: T210381: [cirrus] Add temp clusters but still write to the old ones 2/2 (duration: 00m 46s) | [production] | 
            
  | 12:42 | <dcausse@deploy1001> | Synchronized wmf-config/CommonSettings.php: T210381: [cirrus] Add temp clusters but still write to the old ones 1/2 (duration: 00m 46s) | [production] | 
            
  | 12:31 | <fsero> | pooling mathoid and citoid again on  codfw | [production] | 
            
  | 12:30 | <jdrewniak@deploy1001> | Synchronized portals: Wikimedia Portals Update: [[gerrit:477768| Bumping portals to master (T202497)]] (duration: 00m 46s) | [production] | 
            
  | 12:29 | <jdrewniak@deploy1001> | Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:477768| Bumping portals to master (T202497)]] (duration: 00m 49s) | [production] | 
            
  | 12:23 | <dcausse@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: T211188: Increase autoconfirmed count for Meta-Wiki to 5 (duration: 00m 47s) | [production] | 
            
  | 12:15 | <fsero> | upgrading codfw k8s cluster to 1.10.11 | [production] | 
            
  | 12:15 | <dcausse> | running namespaceDupes & cirrus indexNamespaces on yuewiktionary | [production] | 
            
  | 12:11 | <dcausse@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: T205546: Define 2 new namespaces for yuewiktionary (duration: 00m 47s) | [production] | 
            
  | 12:02 | <fsero> | depooling mathoid and citoid servers on codfw for k8s upgrade | [production] | 
            
  | 11:28 | <mobrovac@deploy1001> | Finished deploy [citoid/deploy@b10e034]: Truncate Zotero-reported time stamp to date - T211127 (duration: 05m 55s) | [production] | 
            
  | 11:23 | <mobrovac@deploy1001> | Started deploy [citoid/deploy@b10e034]: Truncate Zotero-reported time stamp to date - T211127 | [production] | 
            
  | 11:05 | <akosiaris> | upgrade kubernetes-client and kubernetes-master on staging to 1.10.11 | [production] | 
            
  | 11:04 | <godog> | bootstrap cassandra-b, restbase2017 - T210843 | [production] | 
            
  | 10:59 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Repool db1090:3312 (duration: 00m 45s) | [production] | 
            
  | 10:56 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Repool db1090:3317 (duration: 00m 45s) | [production] | 
            
  | 10:51 | <ema> | cache hosts: begin nginx rolling upgrade to 1.13.6-2+wmf2 | [production] | 
            
  | 10:46 | <marostegui> | Reboot db1090 for kernel upgrade | [production] | 
            
  | 10:44 | <moritzm> | uploaded jenkins 2.138.4 to jessie-wikimedia/thirdparty and stretch-wikimedia/thirdpary/ci | [production] | 
            
  | 10:42 | <marostegui> | Stop MySQL on db1090:3312 and db1090:3317 for MySQL upgrade | [production] | 
            
  | 10:42 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Depool db1090:3312 (duration: 00m 46s) | [production] | 
            
  | 10:38 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Depool db1090:3317 (duration: 00m 46s) | [production] | 
            
  | 10:25 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Fully repool db1080 (duration: 00m 46s) | [production] | 
            
  | 10:21 | <gehel@puppetmaster1001> | conftool action : set/pooled=yes; selector: dc=codfw,cluster=elasticsearch | [production] | 
            
  | 10:17 | <arturo> | T205969 icinga downtime the load avg check in labstore1007 for 1 week | [production] |