401-450 of 10000 results (42ms)
2019-03-12 §
13:24 <zfilipin@deploy1001> rebuilt and synchronized wikiversions files: group0 to 1.33.0-wmf.21 [production]
12:40 <arturo> T215605 include python-mwclient .deb in openstack-mitaka-jessie/jessie-wikimedia in install1002 [production]
12:23 <jynus> testing snapshotting on db1117:3325 -> dbstore1001 T210292 [production]
12:23 <zfilipin@deploy1001> Finished scap: testwiki to php-1.33.0-wmf.21 and rebuild l10n cache (duration: 34m 25s) [production]
12:09 <moritzm> upgrading mw1238-mw1258 to component/php72 / PHP 7.2.16 (combined with glibc/OpenSSL updates) [production]
11:59 <mutante> analytics-tool1004 - start superset service [production]
11:48 <zfilipin@deploy1001> Started scap: testwiki to php-1.33.0-wmf.21 and rebuild l10n cache [production]
11:47 <zfilipin@deploy1001> Pruned MediaWiki: 1.33.0-wmf.17 [keeping static files] (duration: 01m 40s) [production]
11:45 <zfilipin@deploy1001> Pruned MediaWiki: 1.33.0-wmf.18 [keeping static files] (duration: 01m 35s) [production]
11:42 <arturo> T215605 include python-oath .deb in stretch-wikimedia thirdparty/oath [production]
11:41 <zfilipin@deploy1001> Pruned MediaWiki: 1.33.0-wmf.16 (duration: 12m 41s) [production]
11:39 <elukey> raise mysql's max_user_connection to 1000 for the Analytics user on labsdb1012 [production]
11:36 <ema@puppetmaster1001> conftool action : set/pooled=yes; selector: name=cp1077.eqiad.wmnet [production]
11:36 <ema> cp1077: repool varnish-be after service restart T217893 [production]
11:35 <arturo> delete wrong stretch-wikimedia `thirdparty` component in install1002 [production]
11:12 <zeljkof> EU SWAT finished [production]
11:11 <kartik@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:495842]] Add campaign prefix for EG tag (T216123) (duration: 00m 49s) [production]
11:11 <moritzm> upgrading API servers/job runners servers in eqiad to component/php72 / PHP 7.2.16 (combined with glibc/OpenSSL updates) (T216712) [production]
10:32 <marostegui> Deploy schema change on db1082, lag will happen on s5 on labs [production]
10:29 <gtirloni> re-enabled puppet on serpens and seaborgium [production]
10:18 <gtirloni> updated slapd to version 2.4.47 on seaborgium (T217280) [production]
10:17 <moritzm> upgrading API servers/job runners servers in codfw to component/php72 / PHP 7.2.16 (combined with glibc/OpenSSL updates) (T216712) [production]
10:14 <gtirloni> upgrading seaborgium to slapd 2.4.47 [production]
09:39 <jynus> stop db1114 and restart it empty [production]
09:25 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1082 (duration: 00m 48s) [production]
08:57 <elukey> restart memcached on mc1019 to apply new settings - T217731 [production]
08:50 <ema> cp1077 depooled again T217893 [production]
08:49 <ema@puppetmaster1001> conftool action : set/pooled=no; selector: name=cp1077.eqiad.wmnet [production]
08:48 <moritzm> upgrading app servers in codfw to component/php72 / PHP 7.2.16 (combined with glibc/OpenSSL updates) (T216712) [production]
08:48 <ema> restart varnish-be on cp1077 T217893 [production]
08:47 <moritzm> upgrading app servers in codfw to component/php72 / PHP 7.2.16 (combined with glibc/OpenSSL updates) [production]
08:46 <ema> cp1077 repooled T217893 [production]
08:46 <ema@puppetmaster1001> conftool action : set/pooled=yes; selector: name=cp1077.eqiad.wmnet [production]
08:36 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1082 for schema change (duration: 00m 48s) [production]
08:34 <jynus> deploy core replica events to db1118 [production]
08:15 <ema> cp1099: ferm.service failed to resolve prometheus1003.eqiad.wmnet. ferm restarted T202966 [production]
07:18 <marostegui> Deploy schema change on db2052 (s5 codfw master), this will generate lag on codfw T71127 T51199 [production]
07:17 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1113 after schema change and upgrade (duration: 00m 49s) [production]
07:09 <marostegui> Upgrade mysql and kernel on db1113 [production]
06:40 <kart_> Finished manual run of unpublished ContentTranslation draft purge script (T217818) [production]
06:17 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1113 for schema change and upgrade (duration: 00m 50s) [production]
04:04 <kart_> Started manual run of unpublished ContentTranslation draft purge script (T217818) [production]
02:40 <ejegg> updated payments-wiki from f1a89d7045 to 7a312e371a [production]
2019-03-11 §
17:55 <addshore@deploy1001> Synchronized wmf-config/interwiki-labs.php: BETA ONLY https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/495723/ (duration: 00m 48s) [production]
17:43 <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings-labs.php: BETA ONLY https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/495721/ (duration: 00m 49s) [production]
17:23 <arturo> T215605 copy python-oath from jessie-wikimedia/thirdparty to stretch-wikimedia/thirdpary in reprepro [production]
17:02 <jbond@cumin1001> END (FAIL) - Cookbook sre.hosts.ipmi-password-reset (exit_code=99) [production]
17:02 <jbond@cumin1001> START - Cookbook sre.hosts.ipmi-password-reset [production]
16:31 <jbond@cumin1001> END (FAIL) - Cookbook sre.hosts.ipmi-password-reset (exit_code=99) [production]
16:31 <jbond@cumin1001> START - Cookbook sre.hosts.ipmi-password-reset [production]