501-550 of 10000 results (19ms)
2020-10-05 §
13:31 <elukey> shutdown an-master1002 for ram expansion (64 -> 128G) [analytics]
12:39 <moritzm> installing curl security updates on remaining hosts [production]
12:35 <elukey> execute "PURGE BINARY LOGS BEFORE '2020-09-28 00:00:00';" on an-coord1001's mysql to free space - T264081 [analytics]
11:39 <hashar> deployment-prep sudo chown jenkins-deploy:wikidev /srv/mediawiki-staging/php-master/.git/index [releng]
11:38 <hashar> deployment-prep: sudo chown -R jenkins-deploy:wikidev /srv/mediawiki-staging/php-master/extensions [releng]
11:37 <hashar> deployment-prep: sudo chown -R jenkins-deploy:wikidev /srv/mediawiki-staging/php-master/extensions/PagedTiffHandler [releng]
11:34 <hoo@deploy1001> Synchronized wmf-config/: Revert "Remove $wgExtraLanguageNames from Wikidata and Commons" (T264295) (duration: 00m 59s) [production]
11:31 <Urbanecm> Restart build of beta-scap-eqiad to sync beta with master branches [releng]
11:30 <Urbanecm> Run cd /srv/mediawiki-staging/php-master/extensions/$1 && git reset HEAD * && git checkout -- * for all extensions at beta (T264563) [releng]
11:28 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: be73f155001e9095697c3c21a208c63e7bf5d2d1: Move changetags right from users to sysop [trwiki] (T264508) (duration: 00m 59s) [production]
11:12 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: cd30b626e23b48146b970c72731f8f7bb1eee9e1: wgSkipSkins: Exclude contenttranslation skin from skin options for users (T263093) (duration: 00m 59s) [production]
11:05 <jdrewniak@deploy1001> Synchronized portals: Wikimedia Portals Update: [[gerrit:632212| Bumping portals to master (T128546)]] (duration: 00m 58s) [production]
11:04 <jdrewniak@deploy1001> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:632212| Bumping portals to master (T128546)]] (duration: 00m 58s) [production]
10:37 <elukey@cumin1001> END (PASS) - Cookbook sre.aqs.roll-restart (exit_code=0) [production]
10:37 <jdrewniak@deploy1001> Synchronized portals: Wikimedia Portals Update: [[gerrit:632204| Bumping portals to master (T128546)]] (duration: 00m 58s) [production]
10:36 <jdrewniak@deploy1001> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:632204| Bumping portals to master (T128546)]] (duration: 01m 00s) [production]
10:34 <elukey@cumin1001> START - Cookbook sre.aqs.roll-restart [production]
10:32 <ema> cp3052: pool with varnish 5.1.3-1wm15 T264398 [production]
10:31 <elukey> bootstrap an-worker111[0,2] as hadoop workers [analytics]
10:31 <elukey> bootstrap an-worker111[0,2 [analytics]
10:28 <ema> cp3052: depool and downgrade varnish to 5.1.3-1wm15 T264398 [production]
10:08 <moritzm> installing ldap-replica1002 T264390 [production]
09:52 <moritzm> installing ldap-replica1001 T264390 [production]
09:22 <moritzm> installing ldap-replica2003 T264390 [production]
09:02 <hnowlan> bootstrapping restbase1030-b [production]
08:57 <moritzm> installing ldap-replica2004 T264390 [production]
08:40 <kormat@cumin1001> dbctl commit (dc=all): 'db2073 depooling: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12918 and previous config saved to /var/cache/conftool/dbconfig/20201005-084022-kormat.json [production]
08:39 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
08:39 <kormat@cumin1001> START - Cookbook sre.hosts.downtime [production]
08:38 <kormat@cumin1001> dbctl commit (dc=all): 'Add db2119 to s4 dump/vslow temporarily T259831', diff saved to https://phabricator.wikimedia.org/P12917 and previous config saved to /var/cache/conftool/dbconfig/20201005-083822-kormat.json [production]
08:23 <godog> prometheus codfw/ops, add 100G to the LV [production]
08:06 <jmm@cumin2001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) [production]
07:46 <marostegui> Stop mysql on es2017 T264386 [production]
07:30 <jmm@cumin2001> START - Cookbook sre.ganeti.makevm [production]
06:52 <XioNoX> add static NAT to pfw3-eqiad - T264356 [production]
06:33 <elukey> reboot stat1005 to resolve weird GPU state (scheduled last week) [analytics]
06:33 <elukey> reboot stat1005 to resolve weird GPU state (scheduled last week) [production]
05:06 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool es2017 T264386 ', diff saved to https://phabricator.wikimedia.org/P12916 and previous config saved to /var/cache/conftool/dbconfig/20201005-050636-marostegui.json [production]
04:57 <legoktm> restarting to deploy T264557 [codesearch]
2020-10-04 §
22:30 <wm-bot> <lucaswerkmeister> php maintenance/createAndPromote.php --force --interface-admin --sysop --bureaucrat "Lucas Werkmeister" [tools.notwikilambda]
14:03 <Urbanecm> tools.stewardbots@tools-sgebastion-07:~$ bash stewardbots/SULWatcher/restart_SULWatcher.sh [tools.stewardbots]
2020-10-03 §
16:42 <wm-bot> <jeanfred> Deploy latest from Git master: e23394e, 567786e, 2316e75, [tools.wikiloves]
16:24 <Reedy> Reloading Zuul to deploy https://gerrit.wikimedia.org/r/631931 [releng]
15:52 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: emergency: 840545f1d9115ea6b672cecce1762d850d8b1f54: Restrict flow-hide right to autoconfirmed users on zhwiki (T264489) (duration: 01m 17s) [production]
10:35 <joal> Manually run mediawiki-history-denormalize after fail-rerun problem (second time) [analytics]
00:08 <ejegg> updated fundraising CiviCRM from 256adda03c to a30da7f92a [production]
2020-10-02 §
22:00 <mutante> depooling mw2271 because Icinga alerts about memcached and SAL shows there were ongoing tests of some kind on it [production]
21:59 <dzahn@cumin1001> conftool action : set/pooled=no; selector: dc=codfw,name=mw2271.codfw.wmnet [production]
21:35 <dzahn@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
21:32 <dzahn@cumin1001> START - Cookbook sre.dns.netbox [production]