|
2016-05-16
§
|
| 14:45 |
<mobrovac> |
change-propagation deployed ef5f6ff55 |
[production] |
| 14:45 |
<bblack> |
starting varnish package upgrade on cache_maps |
[production] |
| 14:39 |
<bblack> |
general package updates on cache clusters (4.4.0 image, libtasn6-1, libjansson4, libidn11) |
[production] |
| 11:20 |
<jynus> |
testing create table on s7-master T130700 |
[production] |
| 10:08 |
<jynus> |
applying puppet and restarting sanitarium instances to apply new filter (temporary labs lag) |
[production] |
| 09:37 |
<volans> |
Slowly re-enabling Puppet on db*, es*, pc*, labsdb*, labservices*, silver, holmium after merged change 288361 T133780 |
[production] |
| 09:31 |
<volans> |
restarted mysql on db2034 to test merged change 288361 , T133780 |
[production] |
|
2016-05-13
§
|
| 23:54 |
<catrope@tin> |
Synchronized php-1.28.0-wmf.1/extensions/Echo/includes/api/ApiEchoNotifications.php: Do not reuse CentralAuth tokens (duration: 00m 25s) |
[production] |
| 22:54 |
<catrope@tin> |
Synchronized php-1.28.0-wmf.1/extensions/Echo/includes/api/ApiEchoNotifications.php: Logging live hack (duration: 00m 31s) |
[production] |
| 22:50 |
<catrope@tin> |
Synchronized php-1.28.0-wmf.1/extensions/Echo/includes/api/ApiEchoNotifications.php: More logging (duration: 00m 25s) |
[production] |
| 22:38 |
<catrope@tin> |
Synchronized php-1.28.0-wmf.1/extensions/Flow/includes/Notifications/Formatter.php: Fix fatal for old Flow notifications (duration: 00m 26s) |
[production] |
| 22:34 |
<catrope@tin> |
Synchronized php-1.28.0-wmf.1/extensions/Echo/: Log failures to fetch cross-wiki notifications (duration: 00m 41s) |
[production] |
| 21:55 |
<catrope@tin> |
Synchronized php-1.28.0-wmf.1/extensions/Echo/Echo.php: Bump cache version now that cache pollution is hopefully fixed (duration: 00m 25s) |
[production] |
| 21:52 |
<catrope@tin> |
Synchronized php-1.28.0-wmf.1/extensions/Echo/: Fixes for cross-wiki notifications deployment fallout (duration: 00m 38s) |
[production] |
| 21:17 |
<bblack> |
cache_misc varnish3 downgrade complete (except 3007 + 1045 - those remain depooled/downtimed/puppet-disabled/etc, do not touch) |
[production] |
| 21:06 |
<catrope@tin> |
Synchronized wmf-config/CommonSettings.php: Enable $wgEchoCrossWikiNotifications on the right wikis (unused for now) (duration: 00m 28s) |
[production] |
| 20:09 |
<mattflaschen@tin> |
Synchronized wmf-config: Disable cross-wiki notifications entirely on non-SUL wikis and hide preference (duration: 00m 34s) |
[production] |
| 20:05 |
<bblack> |
starting varnish3 downgrade process for most of cache_misc |
[production] |
| 17:43 |
<godog> |
powercycle ms-be2008, sdb failed |
[production] |
| 15:41 |
<godog> |
bounce graphite-web on graphite1001, debugging carbon-c-relay stalls while sending metrics |
[production] |
| 13:57 |
<jynus> |
installing and testing sqlproxy on dbproxy1005 |
[production] |
| 13:44 |
<hashar> |
Jenkins zmq is all fine |
[production] |
| 13:43 |
<hashar> |
Restarted Jenkins, the zmq plugin daemon did not start |
[production] |
| 13:13 |
<ema> |
depooling cp3007 to try reproducing T134989 |
[production] |
| 11:28 |
<ema> |
repooling cp1061 |
[production] |
| 11:09 |
<moritzm> |
uploaded firejail 0.38-1+wmf1 for trusty-wikimedia to carbon |
[production] |
| 10:27 |
<ema> |
depooling cp1061 trying to reproduce T134989 |
[production] |
| 09:48 |
<godog> |
nodetool cleanup on restbase2006 T132976 |
[production] |
| 09:11 |
<godog> |
start backfilling cassandra metrics from graphite1001 to graphite1003 |
[production] |
| 05:27 |
<jynus> |
reimporting x1 on dbstore2002 |
[production] |
| 02:30 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Fri May 13 02:30:49 UTC 2016 (duration 8m 50s) |
[production] |
| 02:22 |
<mwdeploy@tin> |
scap sync-l10n completed (1.28.0-wmf.1) (duration: 08m 17s) |
[production] |
| 01:48 |
<awight|animal> |
rollback fundraising-tools from 53f6fe635dd8cc451b86788018e53f418b690b00 to e2978024e6f6b6881d087ac5d07e4c40f7374709 |
[production] |
| 00:53 |
<mutante> |
killing multiple icinga-wm processes on neon, running puppet. manually started and by puppet as well? |
[production] |
| 00:34 |
<demon@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: Remaining wikis to wmf.1 |
[production] |
| 00:17 |
<cwd> |
updated payments from b16c3cb0e12aacb49b91974d693c401dcaa9cca9 to 1af7208b4c7387a701f9ce83e33483d1e19213cb |
[production] |
| 00:13 |
<catrope@tin> |
Synchronized php-1.28.0-wmf.1/extensions/Echo/includes/api/ApiEchoNotifications.php: attempt to fix notices/fatals in Echo (duration: 00m 29s) |
[production] |
| 00:11 |
<catrope@tin> |
Synchronized php-1.27.0-wmf.23/extensions/Echo/includes/api/ApiEchoNotifications.php: attempt to fix notices/fatals in Echo (duration: 00m 25s) |
[production] |
|
2016-05-12
§
|
| 23:58 |
<chasemp> |
restart slapd instances. we have some seemingly ldap issues in labs but it may not be the servers, could be bad merged changes. |
[production] |
| 23:54 |
<dereckson@tin> |
Synchronized php-1.28.0-wmf.1/extensions/CirrusSearch/includes/CirrusSearch.php: Adjust textcat data collection for AB test (T121542) (duration: 00m 26s) |
[production] |
| 23:43 |
<dereckson@tin> |
Synchronized wmf-config/CirrusSearch-common.php: Revert "A/B/C test of control vs textcat vs accept-lang + textcat" ([[Gerrit:288545]], 3/3) (duration: 00m 25s) |
[production] |
| 23:43 |
<dereckson@tin> |
Synchronized wmf-config/InitialiseSettings.php: Revert "A/B/C test of control vs textcat vs accept-lang + textcat" ([[Gerrit:288545]], 2/3) (duration: 00m 25s) |
[production] |
| 23:42 |
<dereckson@tin> |
Synchronized tests/cirrusTest.php: Revert "A/B/C test of control vs textcat vs accept-lang + textcat" ([[Gerrit:288545]], 1/3) (duration: 00m 24s) |
[production] |
| 23:33 |
<dereckson@tin> |
Synchronized wmf-config/InitialiseSettings.php: A/B/C test of control vs textcat vs accept-lang + textcat ([[Gerrit:268048]]) (duration: 00m 25s) |
[production] |
| 23:23 |
<dereckson@tin> |
Synchronized wmf-config/CirrusSearch-common.php: A/B/C test of control vs textcat vs accept-lang + textcat ([[Gerrit:268048]]) (duration: 00m 25s) |
[production] |
| 23:22 |
<dereckson@tin> |
Synchronized tests/cirrusTest.php: A/B/C test of control vs textcat vs accept-lang + textcat (no-op) (duration: 00m 25s) |
[production] |