|
2014-10-23
§
|
| 12:09 |
<Krinkle> |
Zuul/Jenkins stuck. Tried various gearman/zuul resets. Restarting Jenkins now. |
[production] |
| 07:25 |
<_joe_> |
restarted hhvm on mw1114, depooled the server |
[production] |
| 05:39 |
<Tim> |
on mw1189 testing some URLs at a high rate, attempting to induce measurable memory leak |
[production] |
| 05:06 |
<Tim> |
reverted unexplained uncomitted modification of palladium:/srv/pybal-config/pybal/eqiad/api which repooled mw1189 |
[production] |
| 03:41 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Thu Oct 23 03:41:49 UTC 2014 (duration 41m 48s) |
[production] |
| 03:06 |
<cscott> |
Synchronized wmf-config/filebackend.php: fix using a file from commons with file name length between 140 and 159 (duration: 00m 20s) |
[production] |
| 03:01 |
<Krinkle> |
git-deploy: Deploying integration/slave-scripts 157ef23 |
[production] |
| 02:32 |
<springle> |
Synchronized wmf-config/db-eqiad.php: repool db1065, warm up (duration: 00m 06s) |
[production] |
| 02:27 |
<LocalisationUpdate> |
completed (1.25wmf4) at 2014-10-23 02:27:48+00:00 |
[production] |
| 02:15 |
<LocalisationUpdate> |
completed (1.25wmf3) at 2014-10-23 02:15:22+00:00 |
[production] |
| 02:10 |
<springle> |
removed old /var/log/ocg.log* on ocg1002, forced a logrotate |
[production] |
| 02:06 |
<springle> |
upgrade reboot db1065 |
[production] |
|
2014-10-22
§
|
| 23:25 |
<demon> |
Synchronized wmf-config/CommonSettings-labs.php: (no message) (duration: 00m 04s) |
[production] |
| 23:25 |
<demon> |
Synchronized wmf-config/CommonSettings.php: (no message) (duration: 00m 04s) |
[production] |
| 23:22 |
<demon> |
Synchronized php-1.25wmf4/extensions/Collection: (no message) (duration: 00m 04s) |
[production] |
| 23:21 |
<demon> |
Synchronized php-1.25wmf3/extensions/Collection: (no message) (duration: 00m 04s) |
[production] |
| 23:08 |
<mutante> |
added jhernandez to wmf LDAP group |
[production] |
| 22:54 |
<bd808> |
forced puppet run on logstash1003 to pick up https://gerrit.wikimedia.org/r/#/c/168199/ |
[production] |
| 22:47 |
<bd808> |
forced puppet run on logstash1002 to pick up https://gerrit.wikimedia.org/r/#/c/168199/ |
[production] |
| 22:42 |
<bd808> |
forced puppet run on logstash1001 to pick up https://gerrit.wikimedia.org/r/#/c/168199/ |
[production] |
| 22:21 |
<bblack> |
depooled amssq42 (esams text) for trusty testing |
[production] |
| 21:38 |
<bd808> |
killed duplicate logstash services running on logstash1001 and restarted |
[production] |
| 21:16 |
<arlolra> |
updated OCG to version e977e2c8ecacea2b4dee837933cc2ffdc6b214cb |
[production] |
| 21:07 |
<bd808> |
forced puppet run on logstash1001 to pick up https://gerrit.wikimedia.org/r/#/c/168182/ |
[production] |
| 20:40 |
<bd808> |
forced puppet run on logstash1001 to pick up https://gerrit.wikimedia.org/r/168089 |
[production] |
| 20:37 |
<mutante> |
powercycling unresponsive ms-be1012 (this happened before, search SAL for hostname) |
[production] |
| 20:25 |
<arlolra> |
updated Parsoid to version 2a8dc85ce676391acd8c6255c4f94250612c9ee2 |
[production] |
| 16:59 |
<andrewbogott> |
reinstalling OS on virt1006 |
[production] |
| 15:57 |
<marktraceur> |
Synchronized wmf-config/InitialiseSettings.php: [SWAT] Send collection logs to logstash. (duration: 00m 05s) |
[production] |
| 15:31 |
<ottomata> |
setting request.required.acks to 2 for mobile and text varnishkafka's (mobile was set to 2 yesterday) |
[production] |
| 15:24 |
<_joe_> |
rolling restart of hhvm appservers, to alleviate memory issues |
[production] |
| 15:09 |
<marktraceur> |
Synchronized wmf-config/InitialiseSettings.php: [SWAT] Pre-render thumbnails on upload on Commons (duration: 00m 05s) |
[production] |
| 15:05 |
<marktraceur> |
Synchronized wmf-config/CommonSettings-labs.php: [SWAT] Re-enable PediaPress POD in production. (duration: 00m 05s) |
[production] |
| 15:05 |
<marktraceur> |
Synchronized wmf-config/CommonSettings.php: [SWAT] Re-enable PediaPress POD in production. (duration: 00m 05s) |
[production] |
| 13:38 |
<godog> |
catch-up swiftrepl sync eqiad -> codfw for commons containers |
[production] |
| 09:41 |
<hashar> |
Zuul/Jenkins in a deadlock |
[production] |
| 09:08 |
<hashar> |
Restarting Jenkins |
[production] |
| 09:07 |
<hashar> |
Jenkins: upgrading gearman-plugin from 0.0.7-1-g3811bb8 to 0.1.0-1-gfa5f083 . Ie bring us to latest version + 1 commit |
[production] |
| 03:54 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Wed Oct 22 03:54:53 UTC 2014 (duration 54m 52s) |
[production] |
| 02:30 |
<LocalisationUpdate> |
completed (1.25wmf4) at 2014-10-22 02:30:07+00:00 |
[production] |
| 02:17 |
<LocalisationUpdate> |
completed (1.25wmf3) at 2014-10-22 02:17:49+00:00 |
[production] |
|
2014-10-21
§
|
| 23:21 |
<maxsem> |
Synchronized php-1.25wmf4/includes/PrefixSearch.php: https://gerrit.wikimedia.org/r/#/c/167982/ (duration: 00m 03s) |
[production] |
| 23:09 |
<maxsem> |
Synchronized php-1.25wmf3/extensions/MobileFrontend/: SWAT (duration: 00m 04s) |
[production] |
| 23:08 |
<maxsem> |
Synchronized php-1.25wmf4/extensions/MobileFrontend/: SWAT (duration: 00m 04s) |
[production] |
| 23:07 |
<maxsem> |
Synchronized php-1.25wmf4/extensions/CentralAuth/: SWAT (duration: 00m 04s) |
[production] |
| 22:48 |
<mutante> |
radium - installed OS, signing puppet cert requests, initial run ... |
[production] |
| 22:42 |
<legoktm> |
manually finished global rename for BonumTV --> Karypal which failed due to page move timeout |
[production] |
| 22:13 |
<andrewbogott> |
deleted unused labs projects: versionview, feeds, datadog, fundraising-awight, simplewiki, mediawiki-custom-de, fundraising, sartoris, wikibits, incubator, wikiversity-sandbox, data4all |
[production] |
| 21:33 |
<K4-7131> |
adjusting payments antifraud filters |
[production] |
| 21:29 |
<ebernhardson> |
Synchronized php-1.25wmf4/extensions/LiquidThreads/api/ApiQueryLQTThreads.php: Bump LQT in 1.25wmf4 (duration: 00m 04s) |
[production] |