I found this script which works great at extracting the information that is stored in the plist file in each of the page folders in the Wiki structure. However grabbing the content out of the page.html file stored in each .page folder was what I was looking to do. I wrote a helper script that recursively copies and runs the script with a few modifications and then exports all the data I wanted to CSV. The script then copies the CSV files to the main export folder and then deletes all the files that it created in the WIki Server structure.

Usage

To use this script you must copy the folder and all three of the scripts inside it to the root level of your Server HD. Each script has a variable you must set, once you have set the initial path of your Wiki Deployment and the base URL structure you need to make the files executable. You can do this by

1

chmod 700 -R /export

this should make the scripts executable. Once done you need to run the run.sh script with sudo. This will trigger the export. This is no where near perfect so I have opened up a GitHub repository for the changes that I have made, and the addition to the helper script that runs these recursively. This also exports content in user blogs as well.

The one challenge I am having is running the script that exports the page.html file content and keeping the encoding at utf-8 so that I don’t get any artifacts or odd characters.

Here are the scripts

Run.sh

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58

#!/bin/bash
 
##### CONFIGURE HERE ########
 
# put your full path to your collaboration files
 
fullpath=/Wiki/wiki/Collaboration
 
##### END CONFIGURATION #####
 
mkdir /export/users
mkdir /export/users/blogs
mkdir /export/groups
mkdir /export/groups/blogs
mkdir /export/groups/wikis
 
for i in `ls $fullpath/Groups`
do
cp /export/export-blog.sh $fullpath/Groups/$i/weblog/
cp /export/export.sh $fullpath/Groups/$i/wiki/
 
# Export Group Wikis
 
cd $fullpath/Groups/$i/wiki/
./export.sh
mkdir /export/groups/wikis/$i
cp $fullpath/Groups/$i/wiki/wikipages.csv /export/groups/wikis/$i/
rm $fullpath/Groups/$i/wiki/wikipages.csv
rm $fullpath/Groups/$i/wiki/export.sh
 
# Export Group Blogs
 
cd $fullpath/Groups/$i/weblog/
./export-blog.sh
mkdir /export/groups/blogs/$i
cp $fullpath/Groups/$i/weblog/wikipages.csv /export/groups/blogs/$i/
rm $fullpath/Groups/$i/weblog/wikipages.csv
rm $fullpath/Groups/$i/weblog/export-blog.sh
 
done
 
for i in `ls $fullpath/Users`
do
 
# Export User Blogs
 
cp /export/export-blog.sh $fullpath/Users/$i/weblog/
 
cd $fullpath/Users/$i/weblog/
./export-blog.sh
mkdir /export/users/blogs/$i
cp $fullpath/Users/$i/weblog/wikipages.csv /export/users/blogs/$i/
rm $fullpath/Users/$i/weblog/wikipages.csv
rm $fullpath/Users/$i/weblog/export-blog.sh
 
done
 
exit 0

export.sh

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80

#!/bin/sh - 
#
# Script to extract data from an Apple WikiServer's data store by querying the
# filesystem itself. Creates a 'wikipages.csv' file that's readable by any
# spreadsheeting application, such as Numbers.app or Microsoft Excel.app.
#
# USAGE:   To use this script, change to the WikiServer's pages directory, then
#          just run this script. A file named wikipages.csv will be created in
#          your current directory. For instance:
#
#              cd /Library/Collaboration/Groups/mygroup/wiki  # dir to work in
#              wikipages2csv.sh                               # run the script
#              cp wikipages.csv ~/Desktop                     # save output
#
# WARNING: Since the WikiServer's files are only accessible as root, this script
#          must be run as root to function. Additionally, this is not extremely
#          well tested, so use at your own risk.
#
# Author:  Meitar Moscovitz
# Date:    Mon Sep 22 15:03:54 EST 2008
 
##### CONFIGURE HERE ########
 
# The prefix to append to generated links. NO SPACES!
WS_URI_PREFIX=http://my-server.example.com/groups/wiki/
 
##### END CONFIGURATION #####
# DO NOT EDIT PAST THIS LINE
#############################
 
WS_CSV_OUTFILE=wikipages.csv
WS_PAGE_IDS_FILE=`mktemp ws-ids.tmp.XXXXXX`
 
function extractPlistValueByKey () {
    head -n \
      $(expr 1 + `grep -n "<key>$1</key>" page.plist | cut -d ':' -f 1`) page.plist | \
        tail -n 1 | cut -d '>' -f 2 | cut -d '<' -f 1
}
 
function linkifyWikiServerTitle () {
    echo $1 | sed -e 's/ /_/g' -e 's/&amp;/_/g' -e 's/&gt;/_/g' -e 's/&lt;/_/g' -e 's/\?//g'
}
 
function formatISO8601date () {
    echo $1 | sed -e 's/T/ /' -e 's/Z$//'
}
 
function csvQuote () {
    echo $1 | grep -q ',' >/dev/null
    if [ $? -eq 0 ]; then # if there are commas in the string
        echo '"'"$1"'"'   # quote the value
    else
        echo "$1"         # just output the as it was received
    fi
}
 
PSTALLY=`ls -l | grep -v ^l | wc -l`
 
if [ $PSTALLY -gt 4 ] ; then
 
ls -d [^w]*.page | \
  sed -e 's/^\([a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]\)\.page$/\1/' > $WS_PAGE_IDS_FILE
 
fi
 
 
 
echo "Title,ID,Date Created,Last Modified,URI,Content" > $WS_CSV_OUTFILE
while read id; do
 
    cd $id.page
    title="$(extractPlistValueByKey title)"
    created_date="$(formatISO8601date $(extractPlistValueByKey createdDate))"
    modified_date="$(formatISO8601date $(extractPlistValueByKey modifiedDate))"
    link=$WS_URI_PREFIX"$id"/`linkifyWikiServerTitle "$title"`.html
    FILE_DATA=`echo $( /bin/cat page.html ) | tr ',' ' '`
    cd ..
    echo `csvQuote "$title"`,$id,$created_date,$modified_date,`csvQuote "$link"`,"$FILE_DATA" >> $WS_CSV_OUTFILE
done < $WS_PAGE_IDS_FILE
rm $WS_PAGE_IDS_FILE

export-blog.sh

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72

#!/bin/sh -
#
# Script to extract data from an Apple WikiServer's data store by querying the
# filesystem itself. Creates a 'wikipages.csv' file that's readable by any
# spreadsheeting application, such as Numbers.app or Microsoft Excel.app.
#
# USAGE:   To use this script, change to the WikiServer's pages directory, then
#          just run this script. A file named wikipages.csv will be created in
#          your current directory. For instance:
#
#              cd /Library/Collaboration/Groups/mygroup/wiki  # dir to work in
#              wikipages2csv.sh                               # run the script
#              cp wikipages.csv ~/Desktop                     # save output
#
# WARNING: Since the WikiServer's files are only accessible as root, this script
#          must be run as root to function. Additionally, this is not extremely
#          well tested, so use at your own risk.
#
# Author:  Meitar Moscovitz
# Date:    Mon Sep 22 15:03:54 EST 2008
 
##### CONFIGURE HERE ########
 
# The prefix to append to generated links. NO SPACES!
WS_URI_PREFIX=http://my-server.example.com/groups/wiki/
 
##### END CONFIGURATION #####
# DO NOT EDIT PAST THIS LINE
#############################
 
WS_CSV_OUTFILE=wikipages.csv
WS_PAGE_IDS_FILE=`mktemp ws-ids.tmp.XXXXXX`
 
function extractPlistValueByKey () {
    head -n \
      $(expr 1 + `grep -n "<key>$1</key>" page.plist | cut -d ':' -f 1`) page.plist | \
        tail -n 1 | cut -d '>' -f 2 | cut -d '<' -f 1
}
 
function linkifyWikiServerTitle () {
    echo $1 | sed -e 's/ /_/g' -e 's/&amp;/_/g' -e 's/&gt;/_/g' -e 's/&lt;/_/g' -e 's/\?//g'
}
 
function formatISO8601date () {
    echo $1 | sed -e 's/T/ /' -e 's/Z$//'
}
 
function csvQuote () {
    echo $1 | grep -q ',' >/dev/null
    if [ $? -eq 0 ]; then # if there are commas in the string
        echo '"'"$1"'"'   # quote the value
    else
        echo "$1"         # just output the as it was received
    fi
}
 
ls -d [^w]*.page | \
  sed -e 's/^\([a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]\)\.page$/\1/' > $WS_PAGE_IDS_FILE
 
echo "Title,ID,Date Created,Last Modified,URI,Content" > $WS_CSV_OUTFILE
while read id; do
 
    cd $id.page
    title="$(extractPlistValueByKey title)"
    created_date="$(formatISO8601date $(extractPlistValueByKey createdDate))"
    modified_date="$(formatISO8601date $(extractPlistValueByKey modifiedDate))"
    link=$WS_URI_PREFIX"$id"/`linkifyWikiServerTitle "$title"`.html
    FILE_DATA=`echo $( /bin/cat page.html ) | tr ',' ' '`
    cd ..
    echo `csvQuote "$title"`,$id,$created_date,$modified_date,`csvQuote "$link"`,"$FILE_DATA" >> $WS_CSV_OUTFILE
done < $WS_PAGE_IDS_FILE
rm $WS_PAGE_IDS_FILE

I needed a way for my server to notify me by email every time there was a potential problem which results in high CPU usage so that I could mitigate that issue quickly. The server monitor and server admin apps do not allow you to monitor CPU usage and Activity monitor is great as long as you are willing to stand in front of your terminal screen all day. I decided to write a script that would alert me when specific processes started running wild.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

#!/bin/bash
 
processToWatch="PasswordService" # in my case I need to watch convert
emailAddress="me@me.com" # this is my main emailaddress
triggerValue=10 # if the CPU use is above 50% send an email. DO NOT USE a DOT or COMMA!
tempFileName=cpulog # some name of the temp file for the ps, grep data
 
ps auxww | grep "$processToWatch" | grep -v grep > /Scripts/Logs/$tempFileName
export LINE
(
read LINE
while [ -n "$LINE" ]
do
set $LINE
read LINE
if [ $(echo "$3" | sed -e 's/\.[0-9]*//g') -gt $triggerValue ]; then
sudo kill -9 $2;
mail -s "CPU message alert for: $processToWatch" $emailAddress <<-END
This is to inform you that the following process: $processToWatch with PID (Process ID) $2 is now using more than your preset $triggerValue value.
 
Process: $processToWatch is using: $3 of CPU
The command used is: $11
END
fi
done
)< /Scripts/Logs/$tempFileName

The above script will notify me of an issue with the PasswordService and alert me. This has worked out great for me since I only care about a specific service on one server at a time. I can set the time variable in the script to warn me at a specific interval and I use a cron job to schedule the task. I usually have it running every 5 minutes. If you need help with the cron job you can refer to my past post on scheduling tasks on servers using Cron.

I have been playing around with two pieces of software that promise to help bring back this lost functionality in an easy to use GUI tool and perhaps even restore a little sanity to running a website on Lion. The first application that I reviewed was called VirtualHostX.

VirtualHostX 3.0 is the easiest way to host and share multiple websites on your Mac. It’s the perfect solution for web designers working on more than one project at a time. (Aren’t we all?) No more nesting folders or asking the programmer across the cubicle for help. With VirtualHostX you can easily create and manage Apache virtual hosts with just a few clicks.

The other feature that I love about this tool is that you can share a private webpage or site that you are working on, that is not publicly available and share it with anyone publicly through a secure password protected connection. This is great if you need to show people updates of your site and their not on the local subnet. This tool allows you to code custom directives (If you need a list you can check out my last post Missing Manual). 

Out of the box this product works with popular platforms like WordPress and it uses the built in Apache that comes with OSX. Alternatively you can even set it to manage any instance of apache on your server.

Lastly you can even backup the changes that it makes to your system so that you can performa  seamless migration or just for your own peace of mind. I love this software and its an amazing alternative to using the Server app.

The other tool that I found that handles Apache administration on 10.7 is WebMon. Webmon does not look as cool as VirtualHostX however it does have greater support for Custom Directives out of the box in the form of GUI interface.

WebMon configures OS X’s built-in web server to support server-side includes, execCGI, PHP, SSL (including support for inserting Intermediate CA certs) and WebDAV, for multiple domains running on the same server.

With WebDAV turned on, your web server acts like an iDisk, allowing you to connect to the WebDAV folder remotely, securely, and directly from the Finder, so you can save, share, and distribute your files and folders. You can also use the WebDAV folder to share your iCal calendars.

WebMon also helps you set up the web server so that you can monitor its log file from a remote machine. WebMon is able to help you monitor any number of web servers from a single remote machine.

With WebMon you can setup and manage SSL Certificates, turn on CGI Support and much much more. This tool certainly restores almost all of the lost functionality . If you run multiple Web Servers than you might also like its built in monitoring service that makes sure that Apache is running soundly on other systems.

The great thing about both of these solutions is that they work well together, so you can use both or one of them but for the beginner web server administrator these tools restore a little more control when it comes to Apache administration.

I hope that you all found this article and walkthrough educational, as always please feel free to interact with me by posting questions and comments and I will answer them as best as I can. If you feel like any of this is wrong or could be improved upon also please leave a comment below, thanks!

The purpose of this entry is to talk about 10.7 server and show you how to accomplish everything that you could accomplish from the Server Admin application through commands using terminal or edits to system files in the operating system. Everything below requires that you be logged in as the root user on the server in order to avoid permission issues.

How to enable PHP 
Run this command to check if PHP is enabled on 10.7 server.

1

cat /etc/apache2/httpd.conf|grep libphp5.so

If the output is

1

LoadModule php5_module libexec/apache2/libphp5.so

and not

1

#LoadModule php5_module libexec/apache2/libphp5.so

then PHP is enabled. If it is the other way around with a # in the beginning of the line you can just edit the httpd.conf file manually with

1

sudo pico /etc/apache2/httpd.conf

and remove the bracket manually and then restart the web server with

1

sudo apachectl restart

Alternatively you can also enable this via a checkbox in the terrible server.app in 10.7.

How to change the default file type 
By default the landing page on all new sites is index.html if you would like to change this or the order in which a webpage searches for the index page then you need to change the default file type.

To do this edit the configuration file appropriate to your site name. Meaning you have to have already configured a site in the 10.7 server.app program once you have a site then you need to edit the site configuration file. If your site was called apple.com then your site configuration would be in /etc/apache2/sites/apple.com.conf or something like that.

You need to edit that file

1

pico /etc/apache2/sites/nameofyoursite.conf

look for the following in the file

1
2
3

<IfModule mod_dir.c>
    DirectoryIndex index.html
</IfModule>

If you want to change the main page to index.php instead of index.html then replace index.html with index.php. If you want to add it as a secondary load page then you can change it to this.

1
2
3

<IfModule mod_dir.c>
    DirectoryIndex index.html index.php
</IfModule>

once done save and restart apache.

1

sudo apachectl restart

How to enable .htaccess 
If you are going to be using mod_rewwrite at all for redirects or pretty permalinks (which is very common now) then you need to have this enabled. Again as stated before you have to have a site setup on the server through the server.app program. Once done locate your configuration file as outlined above and make the following changes.

1

pico /etc/apache2/sites/nameofyoursite.conf

Once your in the file look for something that looks similar to the following.

1
2
3
4
5
6

<Directory "/Users/yourname/Sites/">
     Options Indexes +MultiViews
     AllowOverride All
     Order allow,deny
     Allow from All
</Directory>

It won’t look exactly the same but what you want to do is replace it with what you see above that will enable the .htaccess or mod_rewrite the line of code that actually does this is the “AllowOverride All” command.

How to enable WebDav
To configure WebDAV Sharing for such users, follow these instructions before enabling any WebDAV share points.

Note: The instructions in this article include editing configuration files. You must have root access to edit these files. You should make a backup copy of each file prior to editing it.

This step is optional but highly recommended: Acquire and install a trusted SSL certificate, and use Server App to configure Web Service to use the certificate. You can use the server’s default, self-signed certificate for WebDAV Sharing, but iWork and other applications may warn that the certificate is “invalid”.

You need to edit the following configuration file

1

pico /etc/apache2/httpd_webdavsharing.conf

Find the line “AuthType Digest” change Digest to Basic. This makes WebDAV Sharing use Basic authentication, which is required for Active Directory users.

Now edit this configuration file

1

pico /etc/apache2/webapps/com.apple.webapp.webdavsharing.plist

find these lines

1
2

<key>sslPolicy</key>
<integer>0</integer>

Change the 0 to 1. This makes WebDAV Sharing require SSL, which is the only secure way to use Basic authentication. Advise users to configure the iWork clients on their iOS devices with an “https” WebDAV URL, like: https://example.com/webdav

How to enable the directory listing 
Again as stated before you have to have a site setup on the server through the server.app program. Once done locate your configuration file as outlined above and make the following changes.

You need to edit that file

1

pico /etc/apache2/sites/nameofyoursite.conf

find the words “AllowOverride” in that block where these words are you need to add this line. This line may already be in your file but it may be different simply update it to reflect these changes

1

Options -Indexes FollowSymLinks

How to enable SSI
If you need to use Server Side Includes in your scripts or website files then do the following to enable it.

1

sudo pico /etc/httpd/httpd.conf

look for these lines

1
2

# AddType text/html .shtml
# AddHandler server-parsed .shtml

Uncomment those 2 lines (remove the # in front of each of them). Now look in the same file for the following

1

Options FollowSymLinks

Add “Includes” to the 2nd line so it looks like

1

Options FollowSymLinks Includes

save the file and restart apache

1

sudo apachectl restart

How to enable VHOSTS
VHOSTS or Virtual Hosts enable you to have multiple domain names mapped to the same site or IP address. To enable this edit the httpd.conf file

1

sudo pico /etc/apache2/httpd.conf

find this line

1

#Include /private/etc/apache2/extra/httpd-vhosts.conf

change it to

1

Include /private/etc/apache2/extra/httpd-vhosts.conf

this will effectively enable VHOSTS. Now you should restart apache.

1

sudo apachectl restart

How to enable CGI
Again as stated before you have to have a site setup on the server through the server.app program. Once done locate your configuration file as outlined above and make the following changes.

1

pico /etc/apache2/sites/nameofyoursite.conf

Once your in the file look for something that looks similar to the following.

1

     Options Indexes +MultiViews

It won’t look exactly the same but what need to do is add “-ExecCGI” after “+MultiViews” it should look something like this.

1

     Options Indexes +MultiViews -ExecCGI

This will enable CGI and allow you to run CGI scripts in Apache. Now you should restart apache.

1

sudo apachectl restart

How to enable Logging
This one boggled my mind, by default website logging is not enabled and again there is no way to enable it in the GUI. You will want to have this enabled to catch errors and fix faulty code. To enable this again we are assuming you already have a site configured with the server.app program. Once done locate your configuration file as outlined above and make the following changes.

1

pico /etc/apache2/sites/nameofyoursite.conf

find the line “DocumentRoot”, Under that line paste the following

1
2

CustomLog "/var/log/apache2/access_log" combinedvhost
ErrorLog "/var/log/apache2/error_log"

it should now look like this

1
2
3

DocumentRoot "/path/to/your/website/"
CustomLog "/var/log/apache2/access_log" combinedvhost
ErrorLog "/var/log/apache2/error_log"

Now you should restart apache.

1

sudo apachectl restart

How to add a domain alias
This is a common thing that most web admins do to map domains to a single site. This again has been removed from the functionality of the server.app on 10.7 server but is a pretty easy to add. To enable this again we are assuming you already have a site configured with the server.app program. Once done locate your configuration file as outlined above and make the following changes.

1

pico /etc/apache2/sites/nameofyoursite.conf

in the site definition file, look for a line that says

1
2

ServerName example.com
ServerAlias www.example.com

where example.com is the domain of your site. You can have more than one alias, just separate them by a spaces on the same line like so.

1
2

ServerName example.com
ServerAlias www.example.com alias2.example.com alias3.example.com

Now you should restart apache.

1

sudo apachectl restart

How to restore factory settings to 10.7 Web Service
This one is important. As stated above you should be backing up these config files before you edit them and then making your changes. In the event that something went wrong you can always reset them back to the original settings.

Run this command

1

sudo serveradmin command web:command=restoreFactorySettings

I got this command by calling Apple directly they also suggested restarting the machine after the restore command, once the computer is back up turn off and then turn on web service to ensure it is working propperly.

Conclusion
All of these commands allow you to leverage Apache and accomplish the tasks that were once easy to accomplish with the Server Admin tool in 10.6 server. There are two options here, learn to love the command line or do not upgrade to 10.7 Lion. Apple is streamlining their GUI interfaces for their tools however there is still power under the hood. Do not be afraid to re-configure these systems Apache, PHP and MYSQL can be installed, modified and improved all from the command line and in some cases they work better after you do. Its not time to quit in my opinion its time to roll up our sleeves and start learning the core of what makes an OSX server truly great and that starts with understanding the open source software that comes bundled with them.

I hope that you all found this article and walkthrough educational, as always please feel free to interact with me by posting questions and comments and I will answer them as best as I can. If you feel like any of this is wrong or could be improved upon also please leave a comment below, thanks!

“I just setup a new server and within days we were on a corporate email blacklist, I contacted the company in question and asked why are we on your blacklist, why won’t you deliver our email. They shared with me an email log of thousands of emails being sent from my mail server through several legitimate email accounts. I ensured that my server was not an open relay so I asked these users, if they had indeed sent this many emails in one shot without any kind of unsubscribe link in the footer of their email. They had! I was so shocked, now what do I do?”

This is an uncomfortable and very perilous position. You want to allow your users to send email to get their job done however you as a systems administrator need to comply with the “Can Spam Act” passed by the FCC to ensure that email continues to flow. You also have companies out there who will block you for violating this act as a precaution on their part. All the while your users can not be bothered to learn about proper email procedures.

In my experience the only thing you can do at this point is to limit how many emails are allowed to be sent at any given time. If you are using OSX Server for Mail or Postfix for Sendmail then this walkthrough will talk about how to limit email recipients and stay off those dreaded blacklists.

Here are the basics that you should know, the following are all settings that can be added to the /etc/postfix/main.cf file of your postfix setup.

smtpd_recipient_limit (default 1000) parameter controls how many recipients the SMTP server will take per message delivery request. You can’t restrict this to a to/cc/bcc field – it’s for all recipients. For that you’d have to use a regular expression in header_checks to arbitrarily limit the length of each header to something reasonable.

smtpd_recipient_overshoot_limit (default 1000) The number of recipients that a remote SMTP client can send in excess of the hard limit specified with smtpd_recipient_limit, before the Postfix SMTP server increments the per-session error count for each excess recipient.

smtpd_hard_error_limit (default 20) parameter to know at what number of errors it will disconnect.

So you technically need to consider the 3 values here which affect both inbound & outbound mail. Then there’s the throttling tools.

smtpd_client_recipient_rate_limit (default: 0 no limit) The maximum number of recipient addresses that an SMTP client may specify in the time interval specified via anvil_rate_time_unit (default: 60s -careful adjusting this affects other things)” and note that this is “regardless of whether or not Postfix actually accepts those recipients” Those over will receive a 450 4.7.1 Error: too many recipients from [the.client.ip.address] It’s up to the client to deliver those recipients at some later time.

smtpd_client_connection_rate_limit (default: 0) The maximal number of connection attempts any client is allowed to make to this service per time unit. The time unit is specified with the anvil_rate_time_unit configuration parameter.

smtpd_client_message_rate_limit (default: 0) The maximal number of message delivery requests that any client is allowed to make to this service per time unit, regardless of whether or not Postfix actually accepts those messages. The time unit is specified with the anvil_rate_time_unit configuration parameter.

The purpose of these features are to limit abuse, as opposed to regulating legitimate mail traffic, but I use them that way in order to mitigate spam blacklisting. In my organization we limit the recipients from one email to 25 you can see the code from my sample /etc/postfix/main.cf. If your file does not have these values you can add them to the bottom of the file.

1
2
3
4
5
6
7
8
9
10

smtpd_recipient_limit = 50
smtpd_recipient_overshoot_limit = 51
smtpd_hard_error_limit = 20
smtpd_client_recipient_rate_limit = 50
smtpd_client_connection_rate_limit = 10
smtpd_client_message_rate_limit = 25
default_extra_recipient_limit = 50
duplicate_filter_limit = 50
default_destination_recipient_limit = 50
smtp_destination_recipient_limit = $default_destination_recipient_limit

Once done you need to restart postfix

sudo postfix reload

I hope that you all found this article and walkthrough educational, as always please feel free to interact with me by posting questions and comments and I will answer them as best as I can. If you feel like any of this is wrong or could be improved upon also please leave a comment below, thanks!

Installation & Configuration

1. Download and install the 64-bit 10.6+ version of MYSQL installer package together with the startup files here.

http://dev.mysql.com/downloads/mysql/

2. Mount the Disk Image (I mean open/double-click the DMG file) and install MySQL server by double-clicking the PKG file (in my case mysql-5.5.14-osx10.6-x86_64.pkg) and follow onscreen instructions. ( It will ask for Master password, as it installs MySQL server in /usr/local )

Current latest version is 5.5.14 which I’ll be using to install on my server.

Open the DMG and you will see that the first item is the MySQL software, the 2nd item allows MySQL to start when the Mac is booted and the third is a System Preference that allows start/stop operation and a preference to enable it to start on boot. Run all of these.

Once the installs are done you can start the mysql server right from the System Preferences which has a new preference in the “Other” category called “MySQL” click start and now it is running.

To find the MySQL version from the terminal, type at the prompt

1

/usr/local/mysql/bin/mysql -v

If you got the error: ERROR 2002 (HY000): Can’t connect to local MySQL server through socket ‘/tmp/mysql.sock’

then mysql was not started, go back to the System Preference and start the database.

3. Run the following commands

1
2
3

cd /usr/local/mysql
cp /usr/local/mysql/support-files/my-small.cnf /private/etc/my.cnf
open -e /private/etc/my.cnf

replace “/tmp/mysql.sock” with “/var/mysql/mysql.sock” at two places near the top.
Create a folder called “mysql” (if you don’t already have one) in the /var directory with the right permissions:

1
2
3
4

cd /var
mkdir mysql
sudo chown -R mysql mysql 
sudo chmod 775 mysql

This command will circumvent the dreaded mysql 2002 socket error.

1
2

sudo mkdir /var/mysql
sudo ln -s /tmp/mysql.sock /var/mysql/mysql.sock

4. Create your alias, this is important so that you can run MYSQL queries through the terminal.

1
2

alias mysql /usr/local/mysql/bin/mysql
alias mysqladmin /usr/local/mysql/bin/mysqladmin

optionally you can edit the ~/.profile file to make your aliases (This should be done as root)

1

pico ~/.profile

then add this line below

1

export PATH=/usr/local/mysql/bin:$PATH

*Please note /usr/local/mysql is only symlink to /usr/local/mysql-5.5.14-osx10.6-x86_64 which means when you upgrade to new version symlink will be changed to point to new version but won’t be deleting the older version. However you need to copy your data directory to new location to make sure your existing databases are intact post upgrade.

5. Set the master MYSQL password, there are 2 ways to do this one is a regular way and the other provides additional security and disables all other access

Regular Way

1

mysqladmin -u root password 'yourpasswordhere'

** use the single quotes. Then when login to mysql to test your password

1

mysql -u root -pyourpasswordhere

Secure Way

1
2
3
4
5
6
7
8
9

sudo mysql_secure_installation
 
NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MySQL
SERVERS IN PRODUCTION USE! PLEASE READ EACH STEP CAREFULLY!
In order to log into MySQL to secure it, we'll need the current
password for the root user. If you've just installed MySQL, and
you haven't set the root password yet, the password will be blank,
so you should just press enter here.
Enter current password for root (enter for none):

Go ahead and just hit enter if this is a new installation and no password currently exists, follow the prompts to set up a new root password – this is a root password just for mysql separate from the root password of OS X you should set this.

You also get asked about:

• Removing anonymous users?
• Disallow root login remotely?
• Remove test database and access to it?
• Reload privilege tables now?
• If this is a new installation you can just answer yes to the questions.

Once the root user and password is set, you have to interact with mysql with the username and password, so access via command line is (note that there is no space between -p and the password)

1

mysql -u root -p[password]

Now that you have MYSQL running you need to start an instance or a main profile for MYSQL to run. I have found the easiest way to do this is to install PHPMYADMIN and since most people in my opinion (Again novice to intermediate MYSQL user here) use this great tool to navigate around MYSQL on a daily basis. Here is a brief walkthrough on how to install and configure PHPMYADMIN on 10.7 Lion Server

Installation & Configuration

1. Change the socket location in your PHP configuration by editing the php.ini file. You need to do a search and replace here. Search and replace all instances of

/var/mysql/mysql.sock

with

/tmp/mysql.sock

Once done you should be able to run the following command and it should reflect the new updated values you just applied.

1

grep .default_socket /etc/php.ini

while editing the php.ini file you need to comment out or enable the following extensions.

extension=php_mysql.dll
extension=php_mysqli.dll

To check your work again you can run this command to ensure they are enabled.

1

grep mysql /etc/php.ini|grep ext

Once done restart Apache

1

sudo apachectl restart

2. Download PHPMYADMIN to the default web directory in Lion

http://www.phpmyadmin.net/home_page/index.php

The full path is

/Library/Server/Web/Data/Sites/Default

I put my PHPMYADMIN in a folder called PHP so

/Library/Server/Web/Data/Sites/Default/PHP

and I could then browse to it by going to

http://server.domain.name/PHP/

this is assuming that you have already configured or turned on web services which I will not go into here since it is a very basic step. I will write a more in depth article and how to on the complexities of running an 10.7 web server in the future however.

Run this command on the PHP Config folder

1

chmod o+w /Library/Server/Web/Data/Sites/Default/PHP/config

3. Now we are ready to run the set up by going to

http://localhost/PHP/setup

The new server to be configured is the localhost, click new server and then the only other configurations are the local mysql user and the password.

Add in the username, by default “root” is assumed, add in the password, click on save and you are returned to the previous screen.

Make sure you click on save, then a config.inc.php is now in the /config directory, move this file to the root level of /phpmyadmin and then remove the empty /config directory.

Now going to http://localhost/PHP/ will now allow you to interact with your mysql databases.

I hope that you all found this article and walkthrough educational, as always please feel free to interact with me by posting questions and comments and I will answer them as best as I can. If you feel like any of this is wrong or could be improved upon also please leave a comment below, thanks!

I don’t really know what to say. I never met him, I never knew him. I was and am such a huge fan, I always watched the keynotes and was so inspired that all I wanted to do was to was to work with the technology that Steve inspired. I became a Certified Apple Technician, and went on to get all of Apples Server Certifications. I was hired by a non profit, they use all Apple technology from the computers to servers and everything in between.

I love my job because I love the technology that I work with and I am truly happy with my life and its all because of Apple and so much of Apple was Steve Jobs. He made it possible for me to be happy and enjoy my life and my job. I am so passionate about this technology. The biggest goal of my life was to eventually meet Steve Jobs or even just be in the same room as him.

I got the go ahead to be at the last WWDC but was not able to acquire tickets. It feels like a big blow to know that I will never meet or see in person the man who has made my life what it is today. I feel a big void in my heart and I truly miss him. I guess I am writing this because I wanted someone to know how much he has touched my life even if it was touched indirectly. It has changed me and made me who I am today.

Thanks Steve.

Jon Brown
IT Director
Food & Water Watch

http://www.foodandwaterwatch.org

Huge Apple Fan

http://www.jonsblog.org

P.S. One of my most favorite clips.

One such service is the topic today, Dovecot. Dovecot is integrated with Server Admin, Apples GUI Server Administration tool. You can set two different kind of notifications to trigger here, a quota notification that will send an email out when someone is over a certain percentage of email quota and an email warning them when they have gone over quota. In my experience it takes more than a couple emails to make a user clean up their inbox.

What I wanted was a way to say, send out an email when a user goes over a specified limit and then send an email every ten percent they go over the original limit. When they reach ten percent before their quota is exceeded increase the email notification rate to one email every percent until they reach their quota and then at that time continue to send an email a day until their quota has been reduced. On top of that I wanted it to also notify me of people who have gone over quota so that I can prove to them that they did indeed get the notification. For me a good solution was having all quota notifications CC’d to our help desk which in turn opened a ticket on the behalf of the offender in a sense sending them two emails each time they went over quota. I am going to cover the necessary steps needed to accomplish this task on your OSX Mail server.

** Note what we are about to do will mean that you will no longer be able to use Server Admin to manage email notifications.

1. Locate the Dovecot Configuration file.

1

 cd /etc/dovecot/dovecot.conf

2. Edit the file

1

 sudo pico /etc/dovecot/dovecot.conf

3. Find this line

1

 quota_warning = storage=100%% /usr/libexec/dovecot/quota-exceeded.sh

we are going to modify this line and add the following lines.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

  quota_warning = storage=100%% /usr/libexec/dovecot/quota-exceeded.sh
  quota_warning2 = storage=99%% /usr/libexec/dovecot/quota-exceeded.sh
  quota_warning3 = storage=98%% /usr/libexec/dovecot/quota-exceeded.sh
  quota_warning4 = storage=97%% /usr/libexec/dovecot/quota-exceeded.sh
  quota_warning5 = storage=96%% /usr/libexec/dovecot/quota-exceeded.sh
  quota_warning6 = storage=95%% /usr/libexec/dovecot/quota-exceeded.sh
  quota_warning7 = storage=94%% /usr/libexec/dovecot/quota-exceeded.sh
  quota_warning8 = storage=93%% /usr/libexec/dovecot/quota-exceeded.sh
  quota_warning9 = storage=92%% /usr/libexec/dovecot/quota-exceeded.sh
  quota_warning10 = storage=91%% /usr/libexec/dovecot/quota-exceeded.sh
  quota_warning11 = storage=90%% /usr/libexec/dovecot/quota-exceeded.sh
  quota_warning12 = storage=87%% /usr/libexec/dovecot/quota-warning.sh
  quota_warning13 = storage=85%% /usr/libexec/dovecot/quota-warning.sh
  quota_warning14 = storage=80%% /usr/libexec/dovecot/quota-warning.sh
  quota_warning15 = storage=75%% /usr/libexec/dovecot/quota-warning.sh

What we are saying here is that we are going to send out an email every time someone is over their limit. Here the limit is 75% and every 5% they go over they will get another warning until they get to 90% then the warnings become more frequent one every 1%. Not only that but there are two different messages the quota-warning and the quota-exceeded.

4. We are going to create a new quota-warning.sh file

1
2

cd /usr/libexec/dovecot
sudo pico quota-warning.sh

This is the current default Apple script that triggers the default email created in Server Admin.

1
2
3
4
5
6
7

#!/bin/sh
 
_quota_txt=/etc/mail/quota_warning.txt
 
if [ -e $_quota_txt ]; then
  cat $_quota_txt | /usr/libexec/dovecot/deliver -d $USER
fi

We are going to modify this script to send out an email of our choice and to do so to another recipient so we have a record of users getting notifications.Here is the script that I wrote that does just that.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

#!/bin/bash
 
PERCENT=$1
FROM_SMTP="support@somedomain.com"
FROM="FWW Support <support@somedomain.com>"
TO="FWW Support <mail-server-admini@somedomain.com>"
qwf="/tmp/quota.warning.$$"
 
echo "From: $FROM
To: $USER
Subject: Quota Notification
Content-Type: text/plain; charset="UTF-8"
 
Hello-
This is a warning email that was automatically sent. You are nearing your quota limit. The current quota is 1 GB of storage space per user. However you can store more offline.
 
Q: What can I do now?
 
A: Start backing up your emails and storing them in a folder under the On My Mac heading, this will ensure that your emails will still be stored and it will free up space on your online account.
 
If you need more assistance please contact Jon Brown at 
support@somedomain.com.
 
Thank you for your cooperation!
 
-- Some Organization Mail Server" > $qwf
 
cat $qwf | /usr/sbin/sendmail -f $FROM_SMTP "$USER"
rm -f $qwf
 
echo "From: $USER
To: support@somedomain.com
Subject: Quota Notification
Content-Type: text/plain; charset="UTF-8"
 
Hello  -
 
$USER Is nearing their quota. Please follow these steps.
 
1. Call the user and make sure they understand how to archive their email.
 
2. Explain to the user that they can sort their email by largest size, tell them to discard or remove the largest emails first.
 
3. Ensure that the quota has been reduced in Server Admin, do not increase the quota unless it is an emergency.
 
-- Some Organization Mail Server" > $qwf
 
cat $qwf | /usr/sbin/sendmail -f $FROM_SMTP "support@somedomain.com"
rm -f $qwf
 
exit 0

You must replace the above script with the old script entirely. This will negate the ability to use the text file that Server Admin uses for email notifications but allows you to send the notification to multiple people.

4. We are going to create a new quota-exceeded.sh file

1
2

cd /usr/libexec/dovecot
sudo pico quota-warning.sh

This is the current default Apple script that triggers the default email created in Server Admin.

1
2
3
4
5
6
7

#!/bin/sh
 
_quota_txt=/etc/mail/quota_exceeded.txt
 
if [ -e $_quota_txt ]; then
  cat $_quota_txt | /usr/libexec/dovecot/deliver -d $USER
fi

We are going to re-write this script and use the following to do similar to the above but at a more aggressive rate.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52

#!/bin/bash
 
PERCENT=$1
FROM_SMTP="support@somedomain.com"
FROM="FWW Support <support@somedomain.com>"
TO="FWW Support <mail-server-admin@somedomain.com>"
qwf="/tmp/quota.warning.$$"
 
echo "From: $FROM
To: $USER
Subject: FWW ***You're Over Your Quota***
Content-Type: text/plain; charset="UTF-8"
 
Hello-
This is a warning email that was automatically sent. You are nearing your quota limit. The current quota is 1 GB of storage space per user. However you can store more offline.
 
Q: What can I do now?
 
A: Start backing up your emails and storing them in a folder under the On My Mac heading, this will ensure that your emails will still be stored and it will free up space on your online account.
 
If you need more assistance please contact Jon Brown at support@somedomain.com.
 
Thank you for your cooperation!
 
-- FWW Mac Server" > $qwf
 
cat $qwf | /usr/sbin/sendmail -f $FROM_SMTP "$USER"
rm -f $qwf
 
echo "From: $USER
To: support@somedomain.com
Subject: FWW ***You're Over Your Quota***
Content-Type: text/plain; charset="UTF-8"
 
Hello  -
 
$USER Is nearing their quota. Please follow these steps.
 
1. Call the user and make sure they understand how to archive their email.
 
2. Explain to the user that they can sort their email by largest size, tell them to discard or remove the largest emails first.
 
3. Ensure that the quota has been reduced in Server Admin, do not increase the quota unless it is an emergency.
 
4. Explain to the user that their email will stop working if they reach 99% capacity.
 
-- FWW Mac Server" > $qwf
 
cat $qwf | /usr/sbin/sendmail -f $FROM_SMTP "support@somedomain.com"
rm -f $qwf
 
exit 0

That is it, once you are done you must restart dovecot.

1
2

sudo serveradmin stop mail
sudo serveradmin start mail

Once done you will now be able to enjoy the fruits of your labor. Your users will now get a lot more notifications which will mean that they will be more likely to tame their unruly inboxes on their own and you will be notified as to when they are getting notifications so that you can better assist them with this task. As always I encourage your comments, suggestions and questions. I hope you all enjoyed my post and thanks for reading!

Again, since I do not have any looming restrictions in my workplace I have found a piece of software that would never be allowed in larger Government facilities but works nicely for what I need. The problem, from time to time I need to re-image or re-core a massive amount of computers, sometimes hundreds of computers. I have a team of two, me and a Helpdesk Technician. This is a daunting task and since I do not like to work weekends, I find that Deploy Studio Server helps me keep my sanity in such situations.

This freeware tool can be used to create deployment files using Netboot, external USB or FireWire drives, or any AFP, SMB, or NFS sharepoint on the network. DeployStudio works with Mac OS X 10.4.11 to 10.6.8 at this point, and is updated regularly to include new OS versions. The package consists of DeployStudio Server, DeployStudio Assistant, DeployStudio Admin, and diffPackageMaker.

DeployStudio Server creates a network based deployment server containing the images. Assistant is used to configure the server and to create the NetInstall sets, while Admin is used to monitor deployments, manage disk images and scripts, enter configurations, and more. diffPackageMaker can look at the difference between two file system snapshots and create installation packages based on what has been changed or added.

I highly recommend using this fine product if you are in the fortunate position as myself and you are not under any pressure or regulations. This requires the use of an in-house server and it installs itself as a service on it. You configure the service to deploy images that you create, and the best part is that it can perform common tasks that will save you time after the re-imaging process is completed. Tasks like setting the computer name, setting up local accounts, binding the computer to a directory server and much more. I describe it as Apple Netboot + Apple Automater = Deploy Studio Server. This is a useful tool that I highly recommend. Check out this instructional video that goes over how to set it up and use it.

I use Deploy Studio Server in my workplace and can field any questions you may have regarding its functionality, setup and configuration and ease of use. Write me a comment below and I will be happy to help!

This is quite frustrating and for the System Admins trying to use Network Accounts, waiting for Adobe to fix this issue has been a waiting game that so far has not come to an end. The obvious solution of course that I tell my users is to use Preview instead of Adobe Acrobat Reader to read their PDF files. This does solve their problems in the short term however my users quickly point out that they need Adobe Acrobat Pro which causes the same issue. Since Preview is no substitution for Adobe Acrobat Pro, this poses a real challenge for the user and the System administrator.

I have been scouring the web trying to find a solution and finally I got a break. A user on an Adobe Forum post, posted a temporary fix that worked wonders for my problem. It was so great I wanted to be sure that this solution gets the exposure that it so rightly deserves. You can read the entire post here and the solution below.

“Hi I have seen this issue on Network accounts for quite a while. It also affects Adobe Acrobat Pro and we have come up with a temporary fix until something is done about the issue. The main problem as I understand it is Adobe Reader does not like writing to network locations.

If you are logged in as a network user then your home directory is going to be something like smb://server/home/user which Adobe does not like and causes the app to crash. To get around this issue we have created a small login hook that creates a symlink in ~/Application\ Support/Adobe which redirects the data to /Users/shared which is stored locally on the machine.

Here is the login hook we’re using if it helps anyone.”

1
2
3
4
5
6
7
8
9

#!/bin/sh
 
rm -rf /Network/Servers/yourservername/homes/$1/Library/Application\ Support/Adobe
 
sudo mkdir -p /Users/Shared/$1
sudo chmod -R 777 /Users/Shared/$1
ln -s /Users/Shared/$1 /Network/Servers/yourservername/homes/$1/Library/Application\ Support/Adobe
 
exit

I throw this in /Library/Preferences and call it symlink.sh and then run the following command to setup the login hook

sudo defaults write com.apple.loginwindow LoginHook /Library/Preferences/symlink.sh

You will find the adobe Reader / Pro and other adobe apps will now work with network accounts. Not the nicest solution but a working one.

I can verify that the solution works well. The script runs, creates the appropriate symlinks and then allows the program to continue to function. The files are created locally for the network user. The only drawback to this is that if you have temporary accounts using computers you will need to clean-up these files from time to time. If the users move about from workstation to workstation then the files will be re-created for that user on multiple machines. These are minor inconveniences that are less noticeable for the user and enable them to get their work done while using Network based accounts in OSX. Let us know what your experiences have been, and if this solution works for you!