|
2017-05-03
§
|
| 08:32 |
<elukey> |
added "adapter=MYSQLI" to config.ini to enable LOAD FILE capabilities on piwik (restarted apache2) |
[analytics] |
| 08:24 |
<_joe_> |
deactivating restbase1018-vg for RAID failover and rebuild T163280 |
[production] |
| 08:20 |
<elukey> |
GRANT FILE on *.* to piwik@localhost executed on bohrium (https://piwik.org/faq/troubleshooting/#faq_194) |
[analytics] |
| 08:16 |
<elukey> |
removed 2>&1 from the Piwik cron archive script |
[analytics] |
| 08:12 |
<elukey> |
set Piwik archive cron on bohrium to run every 3600s (rather than 14400) |
[analytics] |
| 08:01 |
<hashar> |
Rolling back Jenkins 2.46.2 -> 2.46.1 - T144106 |
[production] |
| 07:53 |
<hashar> |
Upgrading Jenkins 2.46.1 -> 2.46.2 - T144106 |
[production] |
| 07:42 |
<_joe_> |
rebuilding RAIDs on restbase1018 T163280 |
[production] |
| 07:35 |
<hashar> |
Restarting Nodepool to catch up with python-jenkins 0.4.14 |
[production] |
| 07:35 |
<moritzm> |
updated python-jenkins on labnodepool1001 to 0.4.14 (needed by latest Jenkins LTS) |
[production] |
| 02:48 |
<l10nupdate@naos> |
ResourceLoader cache refresh completed at Wed May 3 02:48:33 UTC 2017 (duration 5m 21s) |
[production] |
| 02:43 |
<l10nupdate@naos> |
scap sync-l10n completed (1.29.0-wmf.21) (duration: 14m 02s) |
[production] |
| 01:41 |
<mutante> |
kubernetes - puppet fails because "E: Unable to locate package cni |
[production] |
|
2017-05-02
§
|
| 23:55 |
<bd808> |
Increased RAM quota to 16384 (T163375) |
[services-testbed] |
| 23:42 |
<TimStarling> |
EtcdConfig changes all reverted |
[production] |
| 23:17 |
<tstarling@puppetmaster1001> |
conftool action : set/@read-only.yaml; selector: name=ReadOnly,scope=eqiad |
[production] |
| 23:07 |
<TimStarling> |
scap pull on mw2017 and mwdebug1001 for etcd testing |
[production] |
| 23:00 |
<TimStarling> |
locking scap on naos for deployment of EtcdConfig https://gerrit.wikimedia.org/r/#/c/351132/ |
[production] |
| 22:57 |
<_joe_> |
upgrading python-conftool across the fleet |
[production] |
| 22:38 |
<mutante> |
gerrit (cobalt/gerrit2001) - deployed firewall change to allow ssh between gerrit servers for clustering, new iptables rules exist now (T152525) |
[production] |
| 21:52 |
<jynus> |
running previously failed alter tables on s3-eqiad T163912 |
[production] |
| 21:33 |
<jynus> |
creating missing math table on bdwikimedia (s3) |
[production] |
| 21:14 |
<hashar> |
Manually cancelled a few mediawiki-core-jsduck-publish and mediawiki-core-doxygen-publish job in Jenkins build queue. They seems to deadlock Jenkins somehow :( |
[releng] |
| 20:04 |
<hashar> |
Restarting Jenkins for plugin rollback |
[production] |
| 19:59 |
<hashar> |
Regenerate jobs selenium-GettingStarted from JJB - T164296 |
[releng] |
| 19:51 |
<hashar> |
Jenkins: rolling back Performance plugin from 2.2 to 2.0 due to an exception / failure to find a junit xml file. T164296 |
[releng] |
| 19:02 |
<hashar> |
Added multichill ( https://github.com/multichill ) to the Wikimedia Github organization |
[releng] |
| 17:51 |
<bblack> |
codfw->eqiad switchback: end-user edge traffic back to normal @ eqiad ( https://gerrit.wikimedia.org/r/#/c/351330/ ) - 10 minute TTL for bulk traffic pattern shift starts now. |
[production] |
| 17:50 |
<mobrovac@naos> |
Finished deploy [restbase/deploy@6adb0f2]: Include displaytitle and page_id in the summary output and bump the content type version - T163729 T164079 (duration: 06m 04s) |
[production] |
| 17:48 |
<papaul> |
new db servers signing puppet certs,salt-key, initial run |
[production] |
| 17:44 |
<mobrovac@naos> |
Started deploy [restbase/deploy@6adb0f2]: Include displaytitle and page_id in the summary output and bump the content type version - T163729 T164079 |
[production] |
| 17:40 |
<END> |
(PASS) - Start MediaWiki jobrunners, videoscalers and maintenance in codfw - t09_start_maintenance (switchdc/volans@neodymium) |
[production] |
| 17:39 |
<mobrovac@naos> |
Finished deploy [restbase/deploy@6adb0f2]: (no justification provided) (duration: 01m 34s) |
[production] |
| 17:38 |
<START> |
- Start MediaWiki jobrunners, videoscalers and maintenance in codfw - t09_start_maintenance (switchdc/volans@neodymium) |
[production] |
| 17:37 |
<mobrovac@naos> |
Started deploy [restbase/deploy@6adb0f2]: (no justification provided) |
[production] |
| 17:37 |
<END> |
(PASS) - Restore the TTL of all the MediaWiki read-write discovery records and cleanup confd stale files - t09_restore_ttl (switchdc/volans@neodymium) |
[production] |
| 17:37 |
<START> |
- Restore the TTL of all the MediaWiki read-write discovery records and cleanup confd stale files - t09_restore_ttl (switchdc/volans@neodymium) |
[production] |
| 17:35 |
<END> |
(PASS) - Set MediaWiki in read-write mode in codfw - t08_stop_mediawiki_readonly (switchdc/volans@neodymium) |
[production] |
| 17:35 |
<MediaWiki> |
read-only period ends at: 2017-05-02 17:35:48.111079 (switchdc/volans@neodymium) |
[production] |
| 17:35 |
<START> |
- Set MediaWiki in read-write mode in codfw - t08_stop_mediawiki_readonly (switchdc/volans@neodymium) |
[production] |
| 17:35 |
<oblivian@puppetmaster1001> |
conftool action : set/val=test; selector: name=ReadOnly,scope=codfw |
[production] |
| 17:33 |
<END> |
(PASS) - Set core DB masters in read-write mode in codfw, ensure masters in eqiad are read-only - t07_coredb_masters_readwrite (switchdc/volans@neodymium) |
[production] |
| 17:33 |
<START> |
- Set core DB masters in read-write mode in codfw, ensure masters in eqiad are read-only - t07_coredb_masters_readwrite (switchdc/volans@neodymium) |
[production] |
| 17:32 |
<END> |
(PASS) - Switch the Redis masters from eqiad to codfw and invert the replication - t06_redis (switchdc/volans@neodymium) |
[production] |
| 17:32 |
<START> |
- Switch the Redis masters from eqiad to codfw and invert the replication - t06_redis (switchdc/volans@neodymium) |
[production] |
| 17:31 |
<END> |
(PASS) - Switch MediaWiki master datacenter and read-write discovery records from eqiad to codfw - t05_switch_datacenter (switchdc/volans@neodymium) |
[production] |
| 17:31 |
<START> |
- Switch MediaWiki master datacenter and read-write discovery records from eqiad to codfw - t05_switch_datacenter (switchdc/volans@neodymium) |
[production] |
| 17:23 |
<END> |
(FAIL) - Switch MediaWiki master datacenter and read-write discovery records from eqiad to codfw - t05_switch_datacenter (switchdc/volans@neodymium) |
[production] |
| 17:23 |
<START> |
- Switch MediaWiki master datacenter and read-write discovery records from eqiad to codfw - t05_switch_datacenter (switchdc/volans@neodymium) |
[production] |
| 17:20 |
<END> |
(PASS) - Switch traffic flow to the appservers from eqiad to codfw - t05_switch_traffic (switchdc/volans@neodymium) |
[production] |