|
2019-05-03
§
|
| 08:27 |
<jynus> |
starting table recompression on new backup source hosts on eqiad and codfw (stop replication) T220572 |
[production] |
| 07:45 |
<ema> |
depool cp4024 and reimage as upload_ats T219967 |
[production] |
| 07:16 |
<ema> |
cp1089: varnish-backend-restart |
[production] |
| 05:32 |
<_joe_> |
restarting varnish backend on cp1077 |
[production] |
| 05:05 |
<oblivian@puppetmaster1001> |
conftool action : set/pooled=no; selector: cluster=parsoid,dc=codfw,name=wtp201[5-6].* |
[production] |
| 04:57 |
<oblivian@puppetmaster1001> |
conftool action : set/pooled=no; selector: cluster=parsoid,dc=codfw,name=wtp20(1[7-9]|20).* |
[production] |
| 04:55 |
<_joe_> |
progressively depooling parsoid servers in codfw to assess load tolerance |
[production] |
| 00:32 |
<mutante> |
powercycling elastic2038 |
[production] |
| 00:10 |
<XioNoX> |
remove static route to 208.80.155.128/25 on cr1/2-eqiad - T193496 |
[production] |
| 00:06 |
<mutante> |
restarting gerrit to pick up config changes for 2 mail threads and lower timeout (gerrit:507852, gerrit: 507853) |
[production] |
|
2019-05-02
§
|
| 22:10 |
<jforrester@deploy1001> |
Synchronized php-1.34.0-wmf.3/extensions/MobileFrontend/resources/dist/mobile.editor.overlay.js: Hot-deploy T222229 to fix VE switching on MobileFrontend (duration: 00m 52s) |
[production] |
| 21:21 |
<thcipriani> |
gerrit back |
[production] |
| 21:20 |
<ejegg> |
updated payments-wiki from aa8dad50e7 to 558427f731 |
[production] |
| 21:19 |
<thcipriani> |
gerrit restart to pick up config changes: https://gerrit.wikimedia.org/r/504973/ and https://gerrit.wikimedia.org/r/507858/ |
[production] |
| 21:00 |
<crusnov@deploy1001> |
Finished deploy [netbox/deploy@bf9aef2]: Upgrade Netbox to 2.5.12 - T222351 (duration: 01m 48s) |
[production] |
| 20:58 |
<crusnov@deploy1001> |
Started deploy [netbox/deploy@bf9aef2]: Upgrade Netbox to 2.5.12 - T222351 |
[production] |
| 20:58 |
<crusnov@deploy1001> |
Finished deploy [netbox/deploy@bf9aef2]: Upgrade Netbox to 2.5.12 - T222351 (duration: 00m 33s) |
[production] |
| 20:57 |
<crusnov@deploy1001> |
Started deploy [netbox/deploy@bf9aef2]: Upgrade Netbox to 2.5.12 - T222351 |
[production] |
| 19:41 |
<ejegg> |
updated CiviCRM from 01c4d15c9a to 5024c968ed |
[production] |
| 19:40 |
<jforrester@deploy1001> |
Synchronized php-1.34.0-wmf.3/resources/src/mediawiki.widgets/mw.widgets.SearchInputWidget.js: Hot-deploy T222329 fix part 2 (duration: 00m 50s) |
[production] |
| 19:39 |
<jforrester@deploy1001> |
Synchronized php-1.34.0-wmf.3/includes/widget/SearchInputWidget.php: Hot-deploy T222329 fix part 1 (duration: 00m 53s) |
[production] |
| 19:31 |
<James_F> |
Shuffled 1.34.0-wmf.3 security patch cee0e569f4 for T222324 into the tip of the upstream branch now it's merged; no-op |
[production] |
| 19:27 |
<thcipriani@deploy1001> |
rebuilt and synchronized wikiversions files: all wikis to 1.34.0-wmf.3 |
[production] |
| 19:03 |
<mutante> |
phab2001 - apt-get autoremove ..removes a single python package not needed anymore |
[production] |
| 19:00 |
<mutante> |
phab1001 - upgrading PHP packages on prod phab server |
[production] |
| 18:59 |
<jynus> |
restart dbstore1001 for upgrade |
[production] |
| 18:33 |
<catrope@deploy1001> |
Synchronized php-1.34.0-wmf.3/extensions/GrowthExperiments/: Don't fatal on deleted pages in 'recent questions' (T222206) (duration: 01m 01s) |
[production] |
| 18:18 |
<catrope@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Enable cirrussearch-request logging to eventgate-analytics on all wikis (T214080) (duration: 00m 58s) |
[production] |
| 18:10 |
<catrope@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Enable SpecialHomepage on cswiki and kowiki (T221266) (duration: 00m 58s) |
[production] |
| 18:09 |
<mutante> |
phab1001 - install package upgrades for bash and cron |
[production] |
| 17:46 |
<sbassett> |
Deployed patch for T222324 (1.34.0-wmf.3) |
[production] |
| 17:45 |
<arlolra@deploy1001> |
Finished deploy [parsoid/deploy@414387b]: Updating Parsoid to 9786781 (duration: 05m 45s) |
[production] |
| 17:39 |
<arlolra@deploy1001> |
Started deploy [parsoid/deploy@414387b]: Updating Parsoid to 9786781 |
[production] |
| 16:42 |
<gehel> |
replaying 30 minutes of eqiad search traffic on codfw - T221121 |
[production] |
| 16:10 |
<jynus> |
restarted dbproxy1005 haproxy, weird connection issue |
[production] |
| 15:42 |
<reedy@deploy1001> |
Synchronized wmf-config/CommonSettings.php: Re-enable account creation on wikitech (duration: 00m 57s) |
[production] |
| 15:40 |
<reedy@deploy1001> |
Synchronized wmf-config/wikitech.php: Invalidate user sessions upon blocking on wikitech (duration: 00m 59s) |
[production] |
| 15:15 |
<chasemp> |
add dsharpe to content admin on wikitech for user blocking |
[production] |
| 12:42 |
<jynus> |
stopping several instances at dbstore1001 to clone them to db1139/40 T220572 |
[production] |
| 12:06 |
<ema> |
swift-proxy rolling restart T222071 |
[production] |
| 12:01 |
<ema> |
restart swift-proxy on ms-fe1005 T222071 |
[production] |
| 10:37 |
<ariel@deploy1001> |
Finished deploy [dumps/dumps@53c9f22]: speed up adds-changes dumps by generating index.html less often. tmep sleep 120 (duration: 00m 15s) |
[production] |
| 10:36 |
<ariel@deploy1001> |
Started deploy [dumps/dumps@53c9f22]: speed up adds-changes dumps by generating index.html less often. tmep sleep 120 |
[production] |
| 10:03 |
<ema> |
pool cp4023 w/ ATS backend T219967 |
[production] |
| 09:41 |
<jynus> |
testing backups on db2102 (increased network and disk usage) T220572 |
[production] |
| 09:07 |
<jynus> |
reboot db2102 T220572 |
[production] |
| 09:02 |
<ema> |
depool cp4023 and reimage as upload_ats T219967 |
[production] |
| 09:02 |
<godog> |
rollout rsyslog upgrade 8.1901.0-1~bpo9+wmf1 to eqiad |
[production] |
| 08:55 |
<jiji@deploy1001> |
Synchronized wmf-config/CommonSettings.php: Send 5% of anonymous users to PHP7.2 - T219150 (duration: 01m 03s) |
[production] |
| 08:49 |
<jijiki> |
Sending more traffic to PHP7.2 - T219150 |
[production] |