3251-3300 of 10000 results (75ms)
2017-12-04 §
15:18 <demon@tin> Synchronized wmf-config/CommonSettings-labs.php: no-op (duration: 00m 45s) [production]
15:01 <godog> reimage restbase1014 - T179422 [production]
14:51 <herron> cutting over all production puppet agents to codfw puppet 4 masters via dns [production]
14:50 <Amir1> deployed backward compatibility of entity compact diff transmit T113468 [production]
14:49 <ladsgroup@tin> Synchronized php-1.31.0-wmf.10/extensions/Wikibase: (no justification provided) (duration: 01m 41s) [production]
14:37 <gehel@tin> Finished deploy [kartotherian/deploy@e166d87]: dummy kartotherian deployment to test udp2log config change - T175242 (duration: 00m 03s) [production]
14:37 <gehel@tin> Started deploy [kartotherian/deploy@e166d87]: dummy kartotherian deployment to test udp2log config change - T175242 [production]
14:30 <elukey> reboot druid100[23] for kernel updates [production]
14:28 <zfilipin@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:394771|Localize sitename and meta NS for wawiktionary (T181782)]] (duration: 00m 46s) [production]
14:01 <elukey> reboot analytics106* (hadoop worker nodes) for kernel+jvm updates - T179943 [production]
13:24 <paravoid> upgrading bast2001 to stretch [production]
13:22 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1105:3312 - T174569 (duration: 00m 45s) [production]
13:20 <marostegui> Deploy schema change on db1105:3312 (s2) - T174569 [production]
13:05 <marostegui> Compress s6 on db1098 - T178359 [production]
12:44 <gehel@tin> Finished deploy [kartotherian/deploy@e166d87]: testing new kartotherian packaging on maps-test2003 (duration: 00m 22s) [production]
12:43 <gehel@tin> Started deploy [kartotherian/deploy@e166d87]: testing new kartotherian packaging on maps-test2003 [production]
11:06 <jdrewniak@tin> Synchronized portals: Wikimedia Portals Update: [[gerrit:394964|Bumping portals to master (T128546)]] (duration: 00m 45s) [production]
11:05 <jdrewniak@tin> Synchronized portals/prod/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:394964|Bumping portals to master (T128546)]] (duration: 00m 45s) [production]
10:37 <jynus@tin> Synchronized wmf-config/db-codfw.php: Repool db2085 (duration: 00m 43s) [production]
10:06 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Fully pool db1096:3316 - T178359wq! (duration: 00m 45s) [production]
09:55 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Remove db1099:3318 from s5 (duration: 00m 44s) [production]
09:51 <godog> bootstrap restbase1012-c - T179422 [production]
09:32 <godog> clear erroneous table metrics from graphite1003 / graphite2002 - T181689 [production]
09:24 <elukey> reboot analytics104* (hadoop worker nodes) for kernel+jvm updates - T179943 [production]
09:19 <jynus> rebooting mariadb at labsdb1005 [production]
09:12 <moritzm> reimaging mw1259 (video scaler) to stretch, will be kept disabled initially (some controlled live tests following) [production]
08:57 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase traffic for db1096:3315 and 3316 - T178359 (duration: 00m 45s) [production]
08:45 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase traffic for db1096:3316 - T178359 (duration: 00m 45s) [production]
08:44 <moritzm> updating tor on radium to 0.3.1.9 [production]
08:41 <moritzm> updating tor packages to 0.3.1.9 [production]
08:30 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase traffic for db1096:3315 and pool db1096:3316 - T178359 (duration: 00m 45s) [production]
08:12 <marostegui@tin> Synchronized wmf-config/db-codfw.php: Pool db1096:3315 - T178359 (duration: 00m 44s) [production]
08:11 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Pool db1096:3315 - T178359 (duration: 00m 45s) [production]
07:53 <moritzm> installing curl security updates [production]
07:17 <marostegui> Compress s1 on db1099 - T178359 [production]
07:08 <marostegui> Stop MySQL on db1044 as it will be decommissioned - T181696 [production]
07:05 <_joe_> playing with puppetdb status for ores2003 (deactivating/reactivating node) [production]
06:40 <marostegui> Stop MySQL on db1098 to clone db1096.s6 - T178359 [production]
06:39 <marostegui@tin> Synchronized wmf-config/db-codfw.php: Remove db1044 from config as it will be decommissioned - T181696 (duration: 00m 45s) [production]
06:38 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Remove db1044 from config as it will be decommissioned - T181696 (duration: 00m 45s) [production]
06:34 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1098 - T178359 (duration: 00m 46s) [production]
06:21 <marostegui> Deploy alter table on s3 master (db1075) without replication - T174569 [production]
02:32 <l10nupdate@tin> scap sync-l10n completed (1.31.0-wmf.10) (duration: 06m 28s) [production]
2017-12-03 §
15:33 <ejegg> disabled CiviCRM bounce processing job [production]
12:17 <akosiaris> empty ganeti1006, it had issues this morning per T181121 [production]
12:06 <marostegui> Fix dbstore1002 replication [production]
07:44 <akosiaris> ran puppet on conf2002, etcdmirror-conftool-eqiad-wmnet got started again [production]
05:11 <andrewbogott> deleting files on labsdb1003 /srv/tmp older than 30 days [production]
03:57 <no_justification> gerrit2001: icinga is flapping on the gerrit process/systemd check, but this is kind of known (not sure why it's doing this all of a sudden). It's not letting me acknowledge it, but it's fine/harmless. Cf T176532 [production]
2017-12-02 §
17:55 <marostegui> Reboot db1096.s5 to pick up the correct innodb_buffer_pool size after finishing compressing s5 - T178359 [production]