|
2017-10-04
§
|
| 10:03 |
<moritzm> |
install libidn security updates |
[production] |
| 10:01 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1087 with low weight (duration: 00m 50s) |
[production] |
| 09:54 |
<_joe_> |
restarting hhvm on canaries (both appservers and canaries) T107128 |
[production] |
| 09:47 |
<moritzm> |
upgrading API app servers mw1189-mw1208 to HHVM 3.18.5 |
[production] |
| 09:45 |
<elukey> |
rolling restart of aqs nodes to pick up the new logstash lvs config |
[production] |
| 09:39 |
<hoo> |
Killed remaining dumpRdf runners on snapshot1007 after two crashed due to the db1087 depool. Auto-restart will do the rest. |
[production] |
| 09:33 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Add db1104 to the config - T172679 (duration: 00m 50s) |
[production] |
| 09:32 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Add db1104 to the config - T172679 (duration: 00m 51s) |
[production] |
| 09:25 |
<moritzm> |
installing ocaml security updates on trusty |
[production] |
| 09:16 |
<marostegui> |
Reboot db1087 to pick up new kernel |
[production] |
| 09:09 |
<_joe_> |
restarting hhvm on mwdebug*, T107128 |
[production] |
| 09:07 |
<marostegui> |
Optimize templatelinks and pagelinks tables on db1097 (s4) - T174509 |
[production] |
| 09:06 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1097 - T174509 (duration: 00m 50s) |
[production] |
| 08:41 |
<akosiaris@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=wtp10([01][0-9]|2[0-4]).eqiad.wmnet |
[production] |
| 08:40 |
<mobrovac@tin> |
Finished deploy [restbase/deploy@8eb758a]: Switch the mobile feeds to the new storage schema and Cassandra 3 (duration: 10m 05s) |
[production] |
| 08:36 |
<moritzm> |
upgrading app servers mw1180-mw1188, mw1209-mw1220 to HHVM 3.18.5 |
[production] |
| 08:30 |
<mobrovac@tin> |
Started deploy [restbase/deploy@8eb758a]: Switch the mobile feeds to the new storage schema and Cassandra 3 |
[production] |
| 08:28 |
<marostegui> |
Stop MySQL on db2010 as it will be decommissioned - T175685 |
[production] |
| 07:30 |
<jynus> |
stopping s5 replication on dbstore1002 and converting wikidatawiki.wb_terms into TokuDB |
[production] |
| 07:11 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Remove db2010 from config as it will be decommissioned - T175685 (duration: 00m 48s) |
[production] |
| 07:10 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Remove db2010 from config as it will be decommissioned - T175685 (duration: 00m 48s) |
[production] |
| 06:53 |
<marostegui> |
Stop MySQL on db1087 to clone db1104 from it - T172679 |
[production] |
| 06:39 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1087 - T172679 (duration: 00m 50s) |
[production] |
| 06:30 |
<marostegui> |
Optimize templatelinks and pagelinks tables on db1066 - T174509 |
[production] |
| 06:27 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Depool db2035 and db2056 - T174509 (duration: 00m 50s) |
[production] |
| 06:21 |
<elukey> |
restart varnish backend on cp3043 |
[production] |
| 06:19 |
<marostegui> |
Optimize pagelinks and templatelinks on dbstore2002 s1 - T174509 |
[production] |
| 06:16 |
<elukey> |
restart varnish backend on cp3041 |
[production] |
| 06:13 |
<marostegui> |
Optimize pagelinks and templatelinks tables on s7 codfw master (db2029) this will generate lag - T174509 |
[production] |
| 05:58 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Repool db2034, db2055 db2062 - T174509 (duration: 00m 51s) |
[production] |
| 03:09 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Wed Oct 4 03:09:34 UTC 2017 (duration 7m 15s) |
[production] |
| 03:02 |
<l10nupdate@tin> |
scap sync-l10n completed (1.31.0-wmf.2) (duration: 15m 20s) |
[production] |
| 02:25 |
<l10nupdate@tin> |
scap sync-l10n completed (1.31.0-wmf.1) (duration: 07m 44s) |
[production] |
| 00:33 |
<ejegg> |
re-enabled fundraising queue consumers |
[production] |
| 00:29 |
<ejegg> |
re-enabled CiviCRM de-dupe jobs |
[production] |
| 00:24 |
<ejegg> |
re-enabled civicrm cron and ingenico jobs |
[production] |
| 00:20 |
<ejegg> |
re-enabled fundraising stats and export jobs |
[production] |
| 00:16 |
<ejegg> |
re-enabled omnimail import jobs |
[production] |
| 00:10 |
<ejegg> |
re-enabled fundraising audit jobs |
[production] |
| 00:07 |
<ejegg> |
re-enabled pending queue consumer |
[production] |
|
2017-10-03
§
|
| 23:52 |
<ejegg> |
stopped all jobs for process-control upgrade |
[production] |
| 23:46 |
<pnorman> |
Start regenerating map tiles on eqiad for z0-z12 - T176252 |
[production] |
| 23:20 |
<ebernhardson@tin> |
Synchronized wmf-config/InitialiseSettings.php: T175971 T176088 Enable RemexHTML on wikitech and eswikiversity (duration: 00m 51s) |
[production] |
| 23:11 |
<ebernhardson@tin> |
Synchronized wmf-config/CirrusSearch-common.php: Pre-configure cirrussearch for 5.5.x upgrade (duration: 00m 51s) |
[production] |
| 22:26 |
<bsitzmann@tin> |
Finished deploy [mobileapps/deploy@82aa7d6]: Update mobileapps to 5dc0c02 (T175762 T177001 T176525 T176517 T176519) (duration: 06m 14s) |
[production] |
| 22:20 |
<bsitzmann@tin> |
Started deploy [mobileapps/deploy@82aa7d6]: Update mobileapps to 5dc0c02 (T175762 T177001 T176525 T176517 T176519) |
[production] |
| 20:29 |
<pnorman> |
Start regenerating map tiles on codfw for z0-z12 - T176252 |
[production] |
| 20:17 |
<demon@tin> |
Synchronized php-1.31.0-wmf.2/extensions/GlobalUserPage/includes/Hooks.php: fix broken thing (duration: 00m 51s) |
[production] |
| 19:58 |
<demon@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: group0 to wmf.2 |
[production] |
| 19:33 |
<demon@tin> |
Synchronized scap/plugins/prep.py: no-op (duration: 00m 51s) |
[production] |