7751-7800 of 10000 results (72ms)
2019-03-12 §
07:18 <marostegui> Deploy schema change on db2052 (s5 codfw master), this will generate lag on codfw T71127 T51199 [production]
07:17 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1113 after schema change and upgrade (duration: 00m 49s) [production]
07:09 <marostegui> Upgrade mysql and kernel on db1113 [production]
06:40 <kart_> Finished manual run of unpublished ContentTranslation draft purge script (T217818) [production]
06:17 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1113 for schema change and upgrade (duration: 00m 50s) [production]
04:04 <kart_> Started manual run of unpublished ContentTranslation draft purge script (T217818) [production]
02:40 <ejegg> updated payments-wiki from f1a89d7045 to 7a312e371a [production]
2019-03-11 §
17:55 <addshore@deploy1001> Synchronized wmf-config/interwiki-labs.php: BETA ONLY https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/495723/ (duration: 00m 48s) [production]
17:43 <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings-labs.php: BETA ONLY https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/495721/ (duration: 00m 49s) [production]
17:23 <arturo> T215605 copy python-oath from jessie-wikimedia/thirdparty to stretch-wikimedia/thirdpary in reprepro [production]
17:02 <jbond@cumin1001> END (FAIL) - Cookbook sre.hosts.ipmi-password-reset (exit_code=99) [production]
17:02 <jbond@cumin1001> START - Cookbook sre.hosts.ipmi-password-reset [production]
16:31 <jbond@cumin1001> END (FAIL) - Cookbook sre.hosts.ipmi-password-reset (exit_code=99) [production]
16:31 <jbond@cumin1001> START - Cookbook sre.hosts.ipmi-password-reset [production]
15:26 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Fully repool db1097 (duration: 00m 48s) [production]
15:16 <mlitn@deploy1001> Synchronized wmf-config/InitialiseSettings-labs.php: Fix syntax for MediaInfo depicts config (beta only) (duration: 00m 49s) [production]
14:55 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: More traffic to db1097 (duration: 00m 49s) [production]
14:43 <moritzm> upgrading mw canaries to PHP 7.2.16 [production]
14:32 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: More traffic to db1097 (duration: 00m 48s) [production]
14:25 <hashar> contint1001: stopping zuul-merger (it is cpu or IO starving the server) [production]
14:21 <moritzm> upgrading mwdebug servers to PHP 7.2.16 [production]
14:11 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Slowly repool db1097 (duration: 00m 47s) [production]
14:09 <moritzm> importing build of PHP 7.2.16 for component/php72 (T216712) [production]
13:58 <marostegui> Upgrade mysql on db1097 [production]
13:28 <arturo> disable active checks in icinga for labtestvirt200[12] (T218023) [production]
13:04 <moritzm> upgrading mwdebug2002 to php 7.2.16 [production]
12:23 <gtirloni> updated slapd to version 2.4.47 on serpens (T217280) [production]
12:05 <gtirloni> updating slapd on serpens/codfw to test possible fix for memory leaks [production]
10:48 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1097 for upgrade and schema change (duration: 00m 48s) [production]
10:44 <jdrewniak@deploy1001> Synchronized portals: Wikimedia Portals Update: [[gerrit:495650| Bumping portals to master (T128546)]] (duration: 00m 49s) [production]
10:43 <jdrewniak@deploy1001> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:495650| Bumping portals to master (T128546)]] (duration: 00m 49s) [production]
09:56 <moritzm> installing chromium security updates on remaining proton hosts [production]
09:44 <moritzm> installing chromium security updates on proton1001 [production]
09:44 <elukey> roll restart of aqs on aqs100* to pick up new druid settings [production]
08:02 <marostegui> Upgrade pc1010 (spare) [production]
07:33 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Fully repool db1099 after upgrade (duration: 00m 48s) [production]
07:32 <marostegui> Upgrade MySQL and kernel on pc2010 (spare) [production]
07:13 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Give more traffic to db1099 after upgrade (duration: 00m 48s) [production]
06:58 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Give more traffic to db1099 after upgrade (duration: 00m 48s) [production]
06:49 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Slowly repool db1099 after upgrade (duration: 00m 52s) [production]
06:38 <kart_> Finished manual run of unpublished ContentTranslation draft purge script (T217818) [production]
06:37 <marostegui> Power cycle mw1280 - server down [production]
06:35 <marostegui> Upgrade mysql and kernel on db1099 [production]
06:34 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1099 for upgrade (duration: 03m 01s) [production]
06:02 <effie> Restarting pdfrender on scb1003 [production]
06:02 <marostegui> Upgrade MySQL on dbstore1004 (s2, s3, s4) [production]
04:00 <kart_> Started manual run of unpublished ContentTranslation draft purge script (T217818) [production]
03:30 <kartik@deploy1001> Finished deploy [cxserver/deploy@101bebd]: Update cxserver to 5a26308 (T216044, T217878) (duration: 04m 01s) [production]
03:26 <kartik@deploy1001> Started deploy [cxserver/deploy@101bebd]: Update cxserver to 5a26308 (T216044, T217878) [production]
2019-03-10 §
22:35 <gtirloni> toolforge stretch: increased nscd group TTL from 60 to 300sec (T217280) [production]