|
2010-03-26
§
|
| 19:44 |
<Rob> |
ignore nagioswarnings for sq11-sq30, they are being decommissioned |
[production] |
| 18:49 |
<Fred> |
removed srv217 from pybal (api) since the server is not ssh-accessible. |
[production] |
| 18:48 |
<Fred> |
removed srv89 from pybal since the server is not ssh-accessible. |
[production] |
| 18:44 |
<Rob> |
removing sq11-sq30 from nagios and node lists |
[production] |
| 18:41 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Bump style version appendix' |
[production] |
| 13:14 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Bump style version appendix' |
[production] |
| 13:13 |
<catrope> |
synchronized php-1.5/extensions/UsabilityInitiative/js/plugins.combined.min.js 'r64216' |
[production] |
| 13:10 |
<catrope> |
synchronized php-1.5/extensions/UsabilityInitiative/js/plugins.combined.min.js 'More fixes' |
[production] |
| 12:59 |
<catrope> |
synchronized php-1.5/extensions/UsabilityInitiative/js/plugins.combined.min.js 'r64215' |
[production] |
| 07:34 |
<tstarling> |
synchronized php-1.5/extensions/CodeReview/ui/CodeRevisionListView.php 'more path' |
[production] |
|
2010-03-25
§
|
| 23:10 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'bump style version appendix' |
[production] |
| 23:08 |
<catrope> |
synchronized php-1.5/extensions/UsabilityInitiative/js/plugins/jquery.wikiEditor.html 'r64195' |
[production] |
| 23:08 |
<tstarling> |
synchronized php-1.5/wmf-config/CommonSettings.php 'config for r63650' |
[production] |
| 21:54 |
<atglenn> |
added nimish to wmf-deployment committers for current usability testing sessions |
[production] |
| 20:28 |
<JeLuF> |
commons and meta now also use mediawiki-based [[robots.txt]] generation, based on robots.php (#22040) |
[production] |
| 20:23 |
<AaronSchulz> |
deployed r64173 (flaggedrevs_alpha) |
[production] |
| 20:00 |
<JeLuF> |
set up svn in /home/wikipedia/conf/httpd |
[production] |
| 17:36 |
<Rob> |
copied meta favicon to outreachwiki |
[production] |
| 13:35 |
<Rob> |
pdf2 become unaccessible, required reboot |
[production] |
| 12:07 |
<RoanKattouw> |
Deployed r64154 around 11:30 UTC |
[production] |
| 12:07 |
<RoanKattouw> |
Andrew restarted morebots, was down |
[production] |
|
2010-03-24
§
|
| 23:11 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Bump style version appendix' |
[production] |
| 23:08 |
<RoanKattouw> |
Running scap to deploy UsabilityInitiative updates |
[production] |
| 22:10 |
<Tim> |
moved mayflower:/home/mwdocs/phase3 to /var/mwdocs to reduce space usage in /home. Updated apache conf and crontab. |
[production] |
| 22:07 |
<mark> |
Sending European traffic back to esams |
[production] |
| 22:01 |
<Rob> |
sq46 was unresponsive and rebooted, now is working fine |
[production] |
| 21:30 |
<Rob> |
secure.wikimedia.org back online |
[production] |
| 20:47 |
<mark> |
Renamed rr.wikimedia.org to text.wikimedia.org |
[production] |
| 19:38 |
<mark> |
Brought esams Sun V20z servers back up - restored rr.esams and upload.esams LVS |
[production] |
| 19:26 |
<Tempodivalse> |
Can I log things too? |
[production] |
| 19:24 |
<mark> |
Bringing lily back up |
[production] |
| 19:07 |
<jeluf> |
synchronized php-1.5/wmf-config/InitialiseSettings.php '#22924 - Enable the Collection extension at el.wikipedia' |
[production] |
| 18:40 |
<jeluf> |
synchronized live-1.5/robots.php |
[production] |
| 18:39 |
<analyst> |
logger url |
[production] |
| 18:37 |
<river> |
delegated 1.24.10.in-addr.arpa to ha-dns-auth.toolserver.org |
[production] |
| 18:17 |
<Rob> |
techdetail: lowered maxclients on singer apache2 and disabled secure site in apache configuration, do not rollback changes to normal until after esams is back online |
[production] |
| 18:16 |
<Rob> |
techlbog back up, secure is NOT up, and is not expected to come back up, until the esams outage is resolved |
[production] |
| 18:01 |
<Rob> |
disabled secure on singer, as everyone trying to use it overloaded both secure and blogs. |
[production] |
| 18:01 |
<Rob> |
Secure is offline |
[production] |
| 17:51 |
<mark> |
http://techblog.wikimedia.org/2010/03/global-outage-cooling-failure-and-dns/ |
[production] |
| 17:50 |
<mark> |
Reduced Negative TTL setting in SOA record of wikimedia.org from 1H to 10M |
[production] |
| 17:40 |
<domas> |
NXDOMAINs with 1h TTL served on all records for short moment |
[production] |
| 17:40 |
<Rob> |
globally (robs bad) |
[production] |
| 17:40 |
<Rob> |
pending techblog post regarding the issue with accessibility in EU |
[production] |
| 17:25 |
<mark> |
DNS scenario dir esams-down was empty, all zone files missing. Fixed. |
[production] |
| 17:18 |
<mark> |
esams down due to overheating issues; moving traffic to Florida |
[production] |
| 14:18 |
<mark> |
csw2-oe12-esams connected to iris's serial console (temp) |
[production] |
| 04:24 |
<andrew> |
synchronized php-1.5/wmf-config/CommonSettings.php |
[production] |
| 04:21 |
<Andrew> |
Running scap to update LiquidThreads beta |
[production] |
| 04:17 |
<Andrew> |
Preparing to update LiquidThreads beta (production) to alpha version. |
[production] |