6001-6050 of 10000 results (71ms)
2018-08-01 §
07:12 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Remove db1052 from config as it will be decommissioned - T199861 (duration: 00m 55s) [production]
07:11 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Remove db1052 from config as it will be decommissioned - T199861 (duration: 00m 56s) [production]
07:09 <marostegui> Remove db1052 from tendril - T199861 [production]
06:59 <elukey> restart eventlogging on eventlog1002 to pick up new logging settings [production]
06:58 <elukey@deploy1001> Finished deploy [eventlogging/analytics@762ca2b]: Deploy https://gerrit.wikimedia.org/r/#/c/eventlogging/+/449422/ (duration: 00m 07s) [production]
06:58 <elukey@deploy1001> Started deploy [eventlogging/analytics@762ca2b]: Deploy https://gerrit.wikimedia.org/r/#/c/eventlogging/+/449422/ [production]
06:47 <marostegui> Deploy schema change on db1087 with replication, this will generate lag on labs:s8 T144010 T51190 T199368 [production]
06:47 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1087 (duration: 00m 55s) [production]
06:41 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Increase traffic db1089 (duration: 00m 55s) [production]
06:39 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1101:3318 (duration: 00m 55s) [production]
06:22 <brion> restarting transcode batch job (errors believed caused by hhvm config restarts) [production]
06:11 <brion> stopping requeueTranscodes.php for now [production]
06:11 <brion> seeing spike in errors on video scalers [production]
05:14 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Slowly repool db1089 (duration: 00m 57s) [production]
05:02 <marostegui> Stop MySQL on db1089 for a binlog format change (and also upgrade kernel) [production]
05:01 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1089 (duration: 00m 57s) [production]
04:47 <marostegui> Stop MySQL on db1052 to copy its content to dbstore1001 - https://phabricator.wikimedia.org/T199861 [production]
04:46 <marostegui> Deploy schema change on db1101:3318 T144010 T51190 T199368 [production]
04:45 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1101:3318 (duration: 01m 06s) [production]
03:17 <l10nupdate@deploy1001> ResourceLoader cache refresh completed at Wed Aug 1 03:17:12 UTC 2018 (duration 10m 24s) [production]
03:06 <l10nupdate@deploy1001> scap sync-l10n completed (1.32.0-wmf.15) (duration: 14m 45s) [production]
02:34 <l10nupdate@deploy1001> scap sync-l10n completed (1.32.0-wmf.14) (duration: 14m 25s) [production]
00:15 <thcipriani@deploy1001> Synchronized php-1.32.0-wmf.15/extensions/TimedMediaHandler/WebVideoTranscode/WebVideoTranscode.php: SWAT: [[gerrit:449622|Workaround for job queue reporting 0 length]] (T200813) (duration: 00m 56s) [production]
00:08 <thcipriani@deploy1001> Synchronized php-1.32.0-wmf.15/extensions/ParserFunctions/includes/ExtParserFunctions.php: SWAT: [[gerrit:449634|Remove & from $mwDefault variable assignment]] T200772 (duration: 00m 57s) [production]
2018-07-31 §
23:55 <thcipriani@deploy1001> Synchronized php-1.32.0-wmf.14/extensions/TimedMediaHandler/WebVideoTranscode/WebVideoTranscode.php: SWAT: [[gerrit:449621|Workaround for job queue reporting 0 length]] (T200813) (duration: 00m 57s) [production]
23:41 <ejegg> updated payments-wiki from 6afaca3de3 to 626ff1cb72 [production]
23:32 <thcipriani@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:449277|Enable $wgCiteResponsiveReferences for Meta-Wiki]] T200707 (duration: 00m 56s) [production]
23:21 <thcipriani@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:449513|Enable TemplateStyles on Meta-Wiki]] T200613 (duration: 00m 57s) [production]
22:56 <eileen> update CiviCRM civicrm revision changed from 24b497072d to a0a7ea846a, config revision is 35a258d015 [production]
21:37 <twentyafterfour@deploy1001> rebuilt and synchronized wikiversions files: group0 wikis to 1.32.0-wmf.15 [production]
21:05 <twentyafterfour@deploy1001> Finished scap: testwikis wikis to 1.32.0-wmf.15 (duration: 44m 49s) [production]
20:20 <twentyafterfour@deploy1001> Started scap: testwikis wikis to 1.32.0-wmf.15 [production]
19:00 <urandom> enabling `unchecked_tombstone_compaction` on enwiki_T_mobile__ng_remaining -- T192689 [production]
18:20 <XioNoX> stopping puppet across the fleet for puppetmaster1001 uplink move [production]
18:04 <XioNoX> stopping pybal on lvs1002 [production]
18:01 <XioNoX> repool lvs1001 [production]
17:53 <XioNoX> stopping pybal on lvs1001 [production]
17:34 <ayounsi@puppetmaster1001> conftool action : set/pooled=yes; selector: dc=eqiad,cluster=dns,service=pdns_recursor,name=chromium.wikimedia.org [production]
17:19 <twentyafterfour> branching 1.32.0-wmf.15 refs T191061 [production]
17:12 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool all hosts in row B - T183585 (duration: 00m 51s) [production]
16:40 <elukey> repool druid1005 after network maintenance [production]
16:26 <ayounsi@puppetmaster1001> conftool action : set/pooled=no; selector: dc=eqiad,cluster=dns,service=pdns_recursor,name=chromium.wikimedia.org [production]
16:25 <godog> repool thumbor100[12] - T183585 [production]
16:25 <elukey> pool eventbus on kafka1002 after network maintenance [production]
16:24 <ayounsi@puppetmaster1001> conftool action : set/pooled=yes; selector: dc=eqiad,cluster=aqs,service=cassandra,name=aqs1004.eqiad.wmnet [production]
16:24 <ayounsi@puppetmaster1001> conftool action : set/pooled=yes; selector: dc=eqiad,cluster=aqs,service=aqs,name=aqs1004.eqiad.wmnet [production]
16:16 <elukey> precautionary restart of eventbus on kafka1002 after network downtime (DNS name res errors, Kafka broker conn issues, etc..) [production]
16:07 <marostegui> Reload dbproxy1003 and dbproxy1008 [production]
16:06 <ema> reboot cp3044 for kernel update [production]
16:01 <ayounsi@puppetmaster1001> conftool action : set/pooled=yes; selector: dc=eqiad,cluster=prometheus,service=prometheus,name=prometheus1004.eqiad.wmnet [production]