|
2016-05-16
§
|
| 16:54 |
<ottomata> |
restarted kafka broker on kafka1020 (about 5 mins ago) |
[production] |
| 16:36 |
<ottomata> |
restarting kafka broker on kafka1018 |
[production] |
| 16:23 |
<ottomata> |
restarting kafka broker on kafka1014 |
[production] |
| 16:15 |
<ottomata> |
restarting broker on kafka1013 |
[production] |
| 15:42 |
<thcipriani@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: Add an alias "ΒΠ" for Project namespace in elwiki [[gerrit:288965]] (duration: 00m 26s) |
[production] |
| 15:36 |
<thcipriani@tin> |
Synchronized wmf-config/CommonSettings.php: SWAT: Enable MathML rendering by default on test, wikidata and dewikibooks PART II [[gerrit:286180]] (duration: 00m 24s) |
[production] |
| 15:35 |
<thcipriani@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: Enable MathML rendering by default on test, wikidata and dewikibooks PART I [[gerrit:286180]] (duration: 00m 26s) |
[production] |
| 15:31 |
<bblack> |
starting slow cache_upload frontend restarts (wipes) for cache size upgrades (~10H process) |
[production] |
| 15:29 |
<thcipriani@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: Add "deletedtext" right to "eliminator" user group on fawiki [[gerrit:288956]] (duration: 00m 26s) |
[production] |
| 15:25 |
<thcipriani@tin> |
Synchronized wmf-config/abusefilter.php: SWAT: Enable blocking feature of AbuseFilter on ptwiktionary [[gerrit:288810]] (duration: 00m 24s) |
[production] |
| 15:14 |
<thcipriani@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: Set meta namespace names for jamwiki [[gerrit:288628]] (duration: 00m 27s) |
[production] |
| 15:09 |
<bblack> |
varnish package upgrade on cache_maps done |
[production] |
| 15:08 |
<thcipriani@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: wgCopyUploadProxy: Vary per datacenter [[gerrit:287095]] (duration: 00m 26s) |
[production] |
| 15:01 |
<hoo@tin> |
Synchronized wmf-config/Wikibase.php: Update Wikidata property blacklist (duration: 00m 25s) |
[production] |
| 14:55 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Remove all mentions to db1027, db2008 and db2009 (duration: 00m 33s) |
[production] |
| 14:54 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Remove all mentions to db1027, db2008 and db2009 (duration: 00m 34s) |
[production] |
| 14:45 |
<mobrovac> |
change-propagation deployed ef5f6ff55 |
[production] |
| 14:45 |
<bblack> |
starting varnish package upgrade on cache_maps |
[production] |
| 14:39 |
<bblack> |
general package updates on cache clusters (4.4.0 image, libtasn6-1, libjansson4, libidn11) |
[production] |
| 11:20 |
<jynus> |
testing create table on s7-master T130700 |
[production] |
| 10:08 |
<jynus> |
applying puppet and restarting sanitarium instances to apply new filter (temporary labs lag) |
[production] |
| 09:37 |
<volans> |
Slowly re-enabling Puppet on db*, es*, pc*, labsdb*, labservices*, silver, holmium after merged change 288361 T133780 |
[production] |
| 09:31 |
<volans> |
restarted mysql on db2034 to test merged change 288361 , T133780 |
[production] |
|
2016-05-13
§
|
| 23:54 |
<catrope@tin> |
Synchronized php-1.28.0-wmf.1/extensions/Echo/includes/api/ApiEchoNotifications.php: Do not reuse CentralAuth tokens (duration: 00m 25s) |
[production] |
| 22:54 |
<catrope@tin> |
Synchronized php-1.28.0-wmf.1/extensions/Echo/includes/api/ApiEchoNotifications.php: Logging live hack (duration: 00m 31s) |
[production] |
| 22:50 |
<catrope@tin> |
Synchronized php-1.28.0-wmf.1/extensions/Echo/includes/api/ApiEchoNotifications.php: More logging (duration: 00m 25s) |
[production] |
| 22:38 |
<catrope@tin> |
Synchronized php-1.28.0-wmf.1/extensions/Flow/includes/Notifications/Formatter.php: Fix fatal for old Flow notifications (duration: 00m 26s) |
[production] |
| 22:34 |
<catrope@tin> |
Synchronized php-1.28.0-wmf.1/extensions/Echo/: Log failures to fetch cross-wiki notifications (duration: 00m 41s) |
[production] |
| 21:55 |
<catrope@tin> |
Synchronized php-1.28.0-wmf.1/extensions/Echo/Echo.php: Bump cache version now that cache pollution is hopefully fixed (duration: 00m 25s) |
[production] |
| 21:52 |
<catrope@tin> |
Synchronized php-1.28.0-wmf.1/extensions/Echo/: Fixes for cross-wiki notifications deployment fallout (duration: 00m 38s) |
[production] |
| 21:17 |
<bblack> |
cache_misc varnish3 downgrade complete (except 3007 + 1045 - those remain depooled/downtimed/puppet-disabled/etc, do not touch) |
[production] |
| 21:06 |
<catrope@tin> |
Synchronized wmf-config/CommonSettings.php: Enable $wgEchoCrossWikiNotifications on the right wikis (unused for now) (duration: 00m 28s) |
[production] |
| 20:09 |
<mattflaschen@tin> |
Synchronized wmf-config: Disable cross-wiki notifications entirely on non-SUL wikis and hide preference (duration: 00m 34s) |
[production] |
| 20:05 |
<bblack> |
starting varnish3 downgrade process for most of cache_misc |
[production] |
| 17:43 |
<godog> |
powercycle ms-be2008, sdb failed |
[production] |
| 15:41 |
<godog> |
bounce graphite-web on graphite1001, debugging carbon-c-relay stalls while sending metrics |
[production] |
| 13:57 |
<jynus> |
installing and testing sqlproxy on dbproxy1005 |
[production] |
| 13:44 |
<hashar> |
Jenkins zmq is all fine |
[production] |
| 13:43 |
<hashar> |
Restarted Jenkins, the zmq plugin daemon did not start |
[production] |
| 13:13 |
<ema> |
depooling cp3007 to try reproducing T134989 |
[production] |
| 11:28 |
<ema> |
repooling cp1061 |
[production] |
| 11:09 |
<moritzm> |
uploaded firejail 0.38-1+wmf1 for trusty-wikimedia to carbon |
[production] |
| 10:27 |
<ema> |
depooling cp1061 trying to reproduce T134989 |
[production] |
| 09:48 |
<godog> |
nodetool cleanup on restbase2006 T132976 |
[production] |
| 09:11 |
<godog> |
start backfilling cassandra metrics from graphite1001 to graphite1003 |
[production] |
| 05:27 |
<jynus> |
reimporting x1 on dbstore2002 |
[production] |