|
2013-11-20
§
|
| 23:37 |
<ori-l> |
puppet.log on terbium: could not set file on ensure: No such file or directory - /usr/local/apache/common/php/extensions/FlaggedRevs/maintenance/wikimedia-periodic-update.sh.puppettmp_8141 at /etc/puppet/manifests/misc/maintenance.pp:155 |
[production] |
| 23:30 |
<yurik> |
synchronized php-1.23wmf3/extensions/ZeroRatedMobileAccess/ |
[production] |
| 23:27 |
<ori-l> |
restarting rsyslog on application servers for I31c76fdde. |
[production] |
| 22:29 |
<bsitu> |
synchronized wmf-config/InitialiseSettings.php 'touch' |
[production] |
| 22:28 |
<bsitu> |
synchronized echowikis.dblist 'Enable Echo and Thanks on dewiki and itwiki' |
[production] |
| 22:23 |
<bsitu> |
synchronized echowikis.dblist 'Enable Echo and Thanks on dewiki and itwiki' |
[production] |
| 22:23 |
<bsitu> |
synchronized wmf-config/InitialiseSettings.php 'Update email footer and help page' |
[production] |
| 22:20 |
<bsitu> |
updated /a/common to {{Gerrit|Id9e3b9a03}}: correct default Echo help page |
[production] |
| 21:47 |
<faidon> |
synchronized wmf-config/CommonSettings.php 'revert CentralNotice CrossWiki Hiding' |
[production] |
| 21:46 |
<faidon> |
updated /a/common to {{Gerrit|I866838bc2}}: Revert "Enable CentralNotice CrossWiki Hiding" |
[production] |
| 21:34 |
<mutante> |
ran checksetup.pl on bugzilla, deploy gerrit 96479, replaces internal errors on WeeklyReport with proper error pages |
[production] |
| 21:12 |
<gwicke> |
deployed Parsoid 20c6afe |
[production] |
| 19:08 |
<yurik> |
synchronized docroot/bits/WikipediaMobileFirefoxOS |
[production] |
| 18:07 |
<akosiaris> |
changed default ferm policy to DROP. antimony/gallium are the first two servers to get it. |
[production] |
| 17:51 |
<^demon|sick> |
gerrit up, forcing replication of all repos |
[production] |
| 17:50 |
<^demon|sick> |
restarting gerrit since replication plugin won't reload |
[production] |
| 17:27 |
<^demon|sick> |
gerrit: disabled replication plugin fix, pending firewall updates. everything's timing out and flooding logs with errors. |
[production] |
| 13:59 |
<hashar> |
jenkins: added slave integration-slave01 with label hasNpm. That is a slave running in labs ( integration-slave01.pmtpa.wmflabs ) |
[production] |
| 12:31 |
<apergos> |
end of an era: shut down ms7 webserver and opening ticket for decom |
[production] |
| 10:14 |
<hashar> |
Jenkins: disabled gearman plugin, Zuul is no more a Gearman server. |
[production] |
| 10:12 |
<hashar> |
restarted again stalled Jenkins |
[production] |
| 10:05 |
<hashar> |
restarted Zuul with non gearman version. |
[production] |
| 10:02 |
<hashar> |
reverting Zuul to last known version: wmf-deploy-20131023 + 6241272...1e3adfd wmf-deploy-20131023 -> master (forced update) |
[production] |
| 09:58 |
<hashar> |
shutting down Zuul and reverting upgrade :-( |
[production] |
| 09:46 |
<hashar> |
restarting Jenkins to update gearman plugin |
[production] |
| 09:38 |
<hashar> |
Zuul upgraded with Gearman support to trigger build in Jenkins. Will monitor for the rest of the morning. |
[production] |
| 09:34 |
<hashar> |
enabling Gearman in Jenkins, making it register with Zuul new version |
[production] |
| 09:34 |
<hashar> |
restarted Zuul with Gearman version |
[production] |
| 09:27 |
<hashar> |
removing all old versions of zuul: rm -fR /usr/local/lib/python2.7/dist-packages/zuul* |
[production] |
| 09:20 |
<hashar> |
gallium temp install of python-pip to be able to ugprade Zuul |
[production] |
| 09:15 |
<hashar> |
Zuul: bumped source code in integration/zuul.git to Gearman based version: 1e3adfd...6241272 labs -> master |
[production] |
| 09:05 |
<hashar> |
stopping Zuul on gallium for upgrading purposes. |
[production] |
| 07:51 |
<apergos> |
rebooting sq37, we had /dev//sdc gone and reappeared as /dev/sdc killing back-end squid |
[production] |
| 07:40 |
<ori> |
synchronized php-1.23wmf3/extensions/WikimediaEvents/modules/ext.wikimediaEvents.moduleStorage.js 'Re-syncing I51d2d6495' |
[production] |
| 07:38 |
<ori> |
synchronized php-1.23wmf4/extensions/WikimediaEvents/modules/ext.wikimediaEvents.moduleStorage.js 'Re-syncing I51d2d6495' |
[production] |
| 06:47 |
<apergos> |
removed on mw* hosts from /tmp all *png/map/err/tif older than an hour, as some tmpfs were full |
[production] |
| 05:55 |
<ori-l> |
/tmp on Apaches filling up with math .pngs; moving some of the oldest away as a stopgap |
[production] |
| 05:04 |
<springle> |
synchronized wmf-config/db-pmtpa.php |
[production] |
| 03:17 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Wed Nov 20 03:17:15 UTC 2013 |
[production] |
| 02:31 |
<LocalisationUpdate> |
completed (1.23wmf3) at Wed Nov 20 02:31:26 UTC 2013 |
[production] |
| 02:15 |
<LocalisationUpdate> |
completed (1.23wmf4) at Wed Nov 20 02:15:34 UTC 2013 |
[production] |
| 01:38 |
<paravoid> |
rebooting ms-be1001, i/o stuck kernel bug |
[production] |
| 01:37 |
<ori> |
synchronized php-1.23wmf4/extensions/WikimediaEvents 'Update WikimediaEvents to I5b8cfe592' |
[production] |
| 01:35 |
<ori> |
synchronized php-1.23wmf3/resources/startup.js 'touch' |
[production] |
| 01:33 |
<ori> |
synchronized php-1.23wmf3/extensions/WikimediaEvents 'Update WikimediaEvents to I5b8cfe592' |
[production] |
| 00:43 |
<csteipp> |
synchronized php-1.23wmf4/extensions/OAuth 'update OAuth to master for last blocker fix' |
[production] |
| 00:33 |
<ori> |
synchronized php-1.23wmf3/resources/startup.js 'touch' |
[production] |
| 00:28 |
<ori> |
synchronized php-1.23wmf3/extensions/MobileFrontend 'Updating MobileFrontend for I3efc1fa64' |
[production] |