|
2010-07-19
§
|
| 20:05 |
<robh> |
synchronized php-1.5/wmf-config/InitialiseSettings.php 'Bug 24448 - w:cs: upload settings' |
[production] |
| 18:58 |
<aZaFred_OSCON> |
started slave on db10 (sync had stopped on July 8th) |
[production] |
| 16:59 |
<Rob> |
upded tfinc email quota cuz i was tired of seeing the alerts to postmaster |
[production] |
| 13:09 |
<catrope> |
synchronized php-1.5/wmf-config/InitialiseSettings.php 'bug 24440: New namespaces for dawikisource' |
[production] |
| 10:20 |
<midom> |
synchronized wmf-deployment/cache/trusted-xff.cdb |
[production] |
| 08:32 |
<catrope> |
synchronized php-1.5/wmf-config/InitialiseSettings.php 'bug 24435: Enable $wgBlockAllowsUTEdit on hiwiki' |
[production] |
| 07:32 |
<Tim> |
changing max retry count on NTP nagios monitoring to avoid constant flapping |
[production] |
| 06:45 |
<Tim> |
restarted ntpd on mchenry, wasn't responding on IPv4 |
[production] |
|
2010-07-18
§
|
| 21:26 |
<domas> |
fixed opensearch caching, unbreaked API |
[production] |
| 21:18 |
<midom> |
synchronized php-1.5/includes/api/ApiMain.php 'fix broken API caching' |
[production] |
| 15:10 |
<mark> |
synchronizing Wikimedia installation... Revision: 69504 |
[production] |
| 14:59 |
<mark> |
Making puppet upgrade wikimedia-raid-utils on all servers |
[production] |
| 12:58 |
<catrope> |
synchronized php-1.5/extensions/WikimediaMessages/WikimediaLicenseTexts.i18n.php 'r69504' |
[production] |
| 12:51 |
<catrope> |
synchronized php-1.5/wmf-config/InitialiseSettings.php 'New groups for hiwiki (bugs 24416, 24417, 24418, 24419)' |
[production] |
| 08:13 |
<midom> |
synchronized php-1.5/wmf-config/db.php 'db13 and db15 go live' |
[production] |
|
2010-07-17
§
|
| 22:01 |
<mark> |
Apparently package wikimedia-raid-utils truncated /etc/sudoers on many hosts; having puppet put a proper sudoers file back on the application servers |
[production] |
| 21:42 |
<mark> |
Fixed gmond mess on search12 |
[production] |
| 21:03 |
<RoanKattouw> |
Importing another batch of ~10,500 files (~22GB) concurrently with the first one |
[production] |
| 19:38 |
<RoanKattouw> |
Import is running in a screen on fenari |
[production] |
| 19:38 |
<RoanKattouw> |
Importing ~10,500 files (aggregate size ~21GB) with importImages.php for [[commons:User:OrdnanceSurveyBot]] |
[production] |
| 18:49 |
<catrope> |
synchronized php-1.5/languages/messages/MessagesLad.php 'r69487' |
[production] |
| 18:48 |
<mark> |
Having puppet install wikimedia-raid-utils on all servers |
[production] |
| 18:12 |
<mark> |
srv278 rebooted for no apparent reason |
[production] |
| 17:21 |
<mark> |
Fixed ganglia mess on search19 |
[production] |
| 16:19 |
<mark> |
Having puppet install NRPE on all internal servers by default |
[production] |
| 15:52 |
<root> |
synchronized php-1.5/wmf-config/lucene.php 'Use fixed LVS service search-pool3 instead of search7 directly' |
[production] |
| 15:31 |
<mark> |
Removed fscking nscd from spence |
[production] |
| 15:20 |
<mark> |
Added internal LVS services to pmtpa.wmnet DNS |
[production] |
| 15:11 |
<mark> |
Doing reboot test of nescio (esams DNS) |
[production] |
| 15:08 |
<mark> |
Fixed puppet exchange of ssh hostkeys which has been broken for a while |
[production] |
| 14:30 |
<mark> |
Truncated all tables in the puppet db on db9 |
[production] |
| 12:30 |
<mark> |
Implemented domain search list in resolv.conf for fenari (use sparingly!) |
[production] |
| 12:24 |
<mark> |
Put /etc/resolv.conf under Puppet management on all servers. Setting timeout option to 3s, to avoid PyBal depools due to 5s timeout when the primary resolver is down. |
[production] |
| 11:38 |
<mark> |
Removed rkhunter and chkrootkit on bayes. What is the point with 2 year old software? Just creating more cron spam? :) |
[production] |
| 10:06 |
<mark> |
Fixed degraded raid on nescio |
[production] |
| 09:53 |
<mark> |
Removed X.org and gdm from bayes. Why was it installed? |
[production] |
| 09:30 |
<domas> |
doing maintenance on db13 and db15 (BIOS, OS, MySQL upgrades, resync of data) |
[production] |
|
2010-07-16
§
|
| 21:05 |
<mark> |
Removed old-style DNS monitoring from Nagios conf.php, now fully Puppet managed |
[production] |
| 21:03 |
<mark> |
Deployed authoritative DNS on nescio.esams, and moved the service IP |
[production] |
| 19:32 |
<mark> |
Deployed PowerDNS recursor on nescio, and moved the 91.198.174.6 service ip to it |
[production] |
| 19:16 |
<mark> |
Installed lucid on new server nescio.esams |
[production] |
| 13:49 |
<mark> |
Fixed entry in upload.wikimedia.org georecord, sent to text squids for a few ip ranges |
[production] |
| 12:27 |
<mark> |
Pointed country code 'eu' (127.0.255.1 or 65281) to esams in geodns; since geobackend uses a signed short, I had to mask it to 0x7ff / 32513 in the director map |
[production] |
| 10:49 |
<mark> |
Shutdown fuchsia for decommissioning |
[production] |
| 07:57 |
<catrope> |
synchronizing Wikimedia installation... Revision: 69416 |
[production] |
| 07:56 |
<RoanKattouw> |
Running scap to deploy r69416 |
[production] |