|
2020-10-05
ยง
|
| 15:50 |
<thcipriani> |
start deployment-mediawiki-07 |
[releng] |
| 15:43 |
<brennen> |
restarting gitlab-test after disabling prometheus metrics |
[releng] |
| 15:15 |
<ppchelko@deploy1001> |
helmfile [codfw] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'production' . |
[production] |
| 14:56 |
<ppchelko@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'production' . |
[production] |
| 14:55 |
<ppchelko@deploy1001> |
helmfile [staging] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . |
[production] |
| 14:52 |
<milimetric> |
disabling drop-el-unsanitized-events timer until https://gerrit.wikimedia.org/r/c/analytics/refinery/+/631804/ is deployed |
[analytics] |
| 14:41 |
<elukey> |
shutdown stat1005 and stat1008 for ram expansion (1005 again) |
[analytics] |
| 14:41 |
<elukey> |
shutdown stat1005 and stat1008 for ram expansion (1005 again) |
[production] |
| 14:36 |
<ppchelko@deploy1001> |
Finished deploy [restbase/deploy@366a543]: T263133 T264035 (duration: 22m 23s) |
[production] |
| 14:25 |
<elukey> |
shutdown an-master1001 for ram expansion |
[production] |
| 14:25 |
<elukey> |
shutdown an-master1001 for ram expansion |
[analytics] |
| 14:13 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@366a543]: T263133 T264035 |
[production] |
| 14:01 |
<filippo@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'citoid' for release 'production' . |
[production] |
| 13:58 |
<filippo@deploy1001> |
helmfile [codfw] Ran 'sync' command on namespace 'citoid' for release 'production' . |
[production] |
| 13:55 |
<filippo@deploy1001> |
helmfile [staging] Ran 'sync' command on namespace 'citoid' for release 'staging' . |
[production] |
| 13:54 |
<elukey> |
shutdown stat1005 for ram upgrade |
[analytics] |
| 13:54 |
<elukey> |
shutdown stat1005 for ram upgrade |
[production] |
| 13:31 |
<elukey> |
shutdown an-master1002 for ram expansion (64 -> 128G) |
[production] |
| 13:31 |
<elukey> |
shutdown an-master1002 for ram expansion (64 -> 128G) |
[analytics] |
| 12:39 |
<moritzm> |
installing curl security updates on remaining hosts |
[production] |
| 12:35 |
<elukey> |
execute "PURGE BINARY LOGS BEFORE '2020-09-28 00:00:00';" on an-coord1001's mysql to free space - T264081 |
[analytics] |
| 11:39 |
<hashar> |
deployment-prep sudo chown jenkins-deploy:wikidev /srv/mediawiki-staging/php-master/.git/index |
[releng] |
| 11:38 |
<hashar> |
deployment-prep: sudo chown -R jenkins-deploy:wikidev /srv/mediawiki-staging/php-master/extensions |
[releng] |
| 11:37 |
<hashar> |
deployment-prep: sudo chown -R jenkins-deploy:wikidev /srv/mediawiki-staging/php-master/extensions/PagedTiffHandler |
[releng] |
| 11:34 |
<hoo@deploy1001> |
Synchronized wmf-config/: Revert "Remove $wgExtraLanguageNames from Wikidata and Commons" (T264295) (duration: 00m 59s) |
[production] |
| 11:31 |
<Urbanecm> |
Restart build of beta-scap-eqiad to sync beta with master branches |
[releng] |
| 11:30 |
<Urbanecm> |
Run cd /srv/mediawiki-staging/php-master/extensions/$1 && git reset HEAD * && git checkout -- * for all extensions at beta (T264563) |
[releng] |
| 11:28 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: be73f155001e9095697c3c21a208c63e7bf5d2d1: Move changetags right from users to sysop [trwiki] (T264508) (duration: 00m 59s) |
[production] |
| 11:12 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: cd30b626e23b48146b970c72731f8f7bb1eee9e1: wgSkipSkins: Exclude contenttranslation skin from skin options for users (T263093) (duration: 00m 59s) |
[production] |
| 11:05 |
<jdrewniak@deploy1001> |
Synchronized portals: Wikimedia Portals Update: [[gerrit:632212| Bumping portals to master (T128546)]] (duration: 00m 58s) |
[production] |
| 11:04 |
<jdrewniak@deploy1001> |
Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:632212| Bumping portals to master (T128546)]] (duration: 00m 58s) |
[production] |
| 10:37 |
<elukey@cumin1001> |
END (PASS) - Cookbook sre.aqs.roll-restart (exit_code=0) |
[production] |
| 10:37 |
<jdrewniak@deploy1001> |
Synchronized portals: Wikimedia Portals Update: [[gerrit:632204| Bumping portals to master (T128546)]] (duration: 00m 58s) |
[production] |
| 10:36 |
<jdrewniak@deploy1001> |
Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:632204| Bumping portals to master (T128546)]] (duration: 01m 00s) |
[production] |
| 10:34 |
<elukey@cumin1001> |
START - Cookbook sre.aqs.roll-restart |
[production] |
| 10:32 |
<ema> |
cp3052: pool with varnish 5.1.3-1wm15 T264398 |
[production] |
| 10:31 |
<elukey> |
bootstrap an-worker111[0,2] as hadoop workers |
[analytics] |
| 10:31 |
<elukey> |
bootstrap an-worker111[0,2 |
[analytics] |
| 10:28 |
<ema> |
cp3052: depool and downgrade varnish to 5.1.3-1wm15 T264398 |
[production] |
| 10:08 |
<moritzm> |
installing ldap-replica1002 T264390 |
[production] |
| 09:52 |
<moritzm> |
installing ldap-replica1001 T264390 |
[production] |
| 09:22 |
<moritzm> |
installing ldap-replica2003 T264390 |
[production] |
| 09:02 |
<hnowlan> |
bootstrapping restbase1030-b |
[production] |
| 08:57 |
<moritzm> |
installing ldap-replica2004 T264390 |
[production] |
| 08:40 |
<kormat@cumin1001> |
dbctl commit (dc=all): 'db2073 depooling: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12918 and previous config saved to /var/cache/conftool/dbconfig/20201005-084022-kormat.json |
[production] |
| 08:39 |
<kormat@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 08:39 |
<kormat@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 08:38 |
<kormat@cumin1001> |
dbctl commit (dc=all): 'Add db2119 to s4 dump/vslow temporarily T259831', diff saved to https://phabricator.wikimedia.org/P12917 and previous config saved to /var/cache/conftool/dbconfig/20201005-083822-kormat.json |
[production] |
| 08:23 |
<godog> |
prometheus codfw/ops, add 100G to the LV |
[production] |
| 08:06 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) |
[production] |