The WD My Book World edition that can be connected to LAN actually runs Linux inside. Log in to the NAS using the web interface. Next go to the “Advanced Mode” -> “Advanced”, tick the tick box in front of “SSH Access” and press submit. Now you can ssh to the NAS as root. The default password is welc0me. Password can be changed using passwd command.
Category: FreeBSD/Unix
Recovering GPT partitions with gpart
If one manages to destroy the GPT scheme partition tables you’ll still have few options to recover them. Since FreeBSD 9.x the default used by the installation CD is GPT. Just grab the FreeBSD installation CD and boot the system using that. When it boot’s up it gives you options to install, shell, and live CD. Select Live CD and log in as root.
Now if you have previously made a backup of the partition table using gpart backup command then you are lucky and most likely few shell commands away from getting the system back online. If not then you have more things to do.
The backup created by gpart backup command looks like this (backup_partition_table.ada0):
GPT 128 1 freebsd-boot 40 128 2 freebsd-ufs 168 5851053944 3 freebsd-swap 5851054112 8388607
Restoring partitions and bootcode from backup:
gpart restore -F ada0 < backup_partition_table.ada0 gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 ada0
In the restore command -F option first flushes completely the existing data from the partition table and then recovers the table from the file.
In case you haven’t saved the partition table then if you have used the installation defaults then the first partition should be the freebsd-boot and the size is most likely 64k then second partition is the root partition and the third is the swap. By default the boot partition is 64K size other partitions depend on your configuration.
gpart create -s gpt ada0 gpart add -s 64k -t freebsd-boot ada0 gpart add -s 2.7T -t freebsd-ufs ada0 gpart add -t freebsd-swap ada0 gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 ada0
Encountered today a problem on one of the mail servers running dovecot that after upgrade wasn’t able to login using imap and dovecot only printed in to maillog that dovecot: imap-login: Cannot map anonymous memory. It seems that this error is produced if in the dovecot.conf file the login_process_size is set to too low value. The explanation and default for the value is:
# Set max. process size in megabytes. If you don't use # login_process_per_connection you might need to grow this. # login_process_size = 64
So what I did is that in the dovecot.conf file I uncommented the login_process_size line and increased it to 256:
# Set max. process size in megabytes. If you don't use # login_process_per_connection you might need to grow this. login_process_size = 256
PHP 5.4.x and UTF-8
As the PHP 5.4 release note says the default character encoding has been changed from ISO-8859-1 (latin1) to UTF-8. Although the encoding can be controlled by setting default_charset = “iso-8859-1” in php.ini. This change doesn’t seem to have an effect on functions like htmlspecialchars/htmlentities which have their own default encoding (UTF-8) and own function argument for character encoding. If you want to continue using 5.4 the only way to solve this issue is to modify all the places where this function is called to following:
$str_to_be_coded = "äÄöÖüÜ";
$str = htmlspecialchars($str_to_be_coded, ENT_COMPAT | ENT_HTML401 | ENT_SUBSTITUTE, ISO-8859-1);
echo $str;
$str = htmlentities($str_to_be_coded, ENT_COMPAT | ENT_HTML401 | ENT_SUBSTITUTE, ISO-8859-1);
echo $str;
If you omit the ENT_SUBSTITUTE then the htmlspecialchars may return null string if the string contains umlauts (äÄöÖüÜ).
The alternate solution is of course to use UTF-8 encoding but this might require (if your system wide setting is ISO-8859-1) you to configure the Apache server to use UTF-8 encoding and you to update your PHP scripts to use UTF-8 encoding inside. In addition if you are using MySQL you should set the character set with MySQL as well:
/* change character set to utf8 */
if (!$mysqli->set_charset("utf8")) {
printf("Error loading character set utf8: %s\n", $mysqli->error);
} else {
printf("Current character set: %s\n", $mysqli->character_set_name());
}
Finally the solution to which I did with one of the third party software was that I installed from FreeBSD ports the package called lang/php53 which contains the latest 5.3 branch PHP interpreter.
Just wanted to share a very short how to on creating GPT partition table using part command. The reasons for using GPT instead of MBR are quite obvious as disk spaces increase. The MBR partition table restricts partition sizes to a maximum of 2.19 terabytes (2.19 × 1012 bytes) or almost exactly 2 TB (2,199,023,255,040 bytes or 4,294,967,295 (232?1) sectors × 512 (29) bytes per sector). GPT supports partition sizes up to 9.3 ZB or 8 ZiB?512 bytes (9,444,732,965,739,290,426,880 bytes or 18,446,744,073,709,551,615 (264?1) sectors × 512 (29) bytes per sector). MBR disks support only four partition table entries. If more partitions are wanted, a secondary structure known as an extended partition is necessary. Extended partitions can then be subdivided into one or more logical disks. Similarly GPT support 128 partitions on a single disk allowing much larger number of filesystems on a single device without doing extension structures.
To create an empty GPT partition table run:
# gpart create -s gpt mfid1
You can check the result:
# gpart show mfid1
=> 34 5857345469 mfid1 GPT (2.7T)
34 5857345469 - free - (2.7T)
Now create first a 50G partition:
# gpart add -t freebsd-ufs -s 50G mfid1
mfid1p1 added
Then second partition that uses rest of the drive
# gpart add -t freebsd-ufs mfid1
mfid1p2 added
And the result:
# gpart show mfid1
=> 34 5857345469 mfid1 GPT (2.7T)
34 104857600 1 freebsd-ufs (50G)
104857634 5752487869 2 freebsd-ufs (2.7T)
Then just initialize the file systems:
# newfs /dev/mfid1p1
...
# newfs /dev/mfid1p2
...
First one should naturally install subversion and apache in to the system and while installing make sure that mod_dav option is enabled in apache package and that mod_dav_svn option is enabled in subversion.
There is two ways to setup the mod_dav_svn with apache depending on whether you plan to have one or multiple repositories within single URI. For single repository the you should add in to httpd.conf the following:
## SVN WebDAV Repository Setup <Location /svn> DAV svn SVNPath /home/svn/repos SVNIndexXSLT "http://www.myservername.com/svnindex.xsl" # authenticating them valid ones Satisfy All Require valid-user AuthType Basic AuthName "Subversion Repositories" AuthUserFile /home/svn/access/users </Location>
For multiple repositories add following in to httpd.conf:
## SVN WebDAV Repository Setup <Location /svn> DAV svn SVNParentPath /home/svn/repos SVNIndexXSLT "http://www.myservername.com/svnindex.xsl" # authenticating them valid ones Satisfy All Require valid-user AuthType Basic AuthName "Subversion Repositories" AuthUserFile /home/svn/access/users </Location>
The default svnindex.xsl and svnindex.css can be found and copied from the subversion source package under subversion-<VERSION-NUMBER>/tools/xslt/.
Create the users using the htpasswd utility when adding the first user add -c after the htpasswd command in order to create the password file and for the following users you should remove the -c option:
# htpasswd /home/svn/access/users <username>
Building repository in the single repository configuration. The repository needs to be writable by the user or group to which the apache process belongs to.
# svnadmin create /home/svn/repos # svn import /home/<where_ever_you_have_initial_files> \ file:///home/svn/repos -m "initial import" # cd /home/svn # chown -R www repos
Building repository in the multiple repository configuration. The repository needs to be writable by the user or group to which the apache process belongs to.
# svnadmin create /home/svn/repos/<PROJECT_NAME> # svn import /home/<where_ever_you_have_initial_files> \ file:///home/svn/repos/<PROJECT_NAME> -m "initial import" # cd /home/svn/repos # chown -R www <PROJECT_NAME>
In the single repository configuration you can access the repository through web browser simply by writing in to the location bar the http://www.myservername.com/svn, but in the multiple repository configuration you must also add the project name after e.g. http://www.myservername.com/svn/<PROJECT_NAME> simply accessing the parent directory will result in to permission denied page to displayed so listing of repositories is not possible.
pf rtables and setfib in FreeBSD
If one has multiple outgoing links to which one would like to use different routing tables the FreeBSD provides possibility through the setfib command but in order to have multiple routing tables one has to first compile a custom kernel with option ROUTETABLES in example simple kernel config:
include GENERIC options ROUTETABLES=4
After the kernel has been built and rebooted the different routing tables can be accessed as shown in the setfib(1) man page by issuing command setfib 0 netstat -rn. 0 is the default routing table.
After this one has to create the second routing table by prepending every route add command with setfib 1 route add… e.g:
# setfib 1 route add -net default 10.0.0.1
With packet filter one can control how the routing table is selected by using rtable option but it should be noted that this selection can only be done on the input of the packets as the routing decision is done at the input not at the output. Here is an example of very simple pf.conf that uses rtable rules and NATs everything to the external interface address:
# # Macros # INT_IF = "em0" EXT_IF = "bge0" EXT_IF2 = "bge1" table <private_nets> persist { 127/8, 172.16/12, 192.168/16, 169.254/16 } # # Options and default policy # set block-policy drop set state-policy if-bound # # Packet normalization # scrub in all scrub out on $EXT_IF all random-id scrub on $EXT_IF all reassemble tcp # # NAT/redirects # # NAT nat on $EXT_IF from <private_nets> to any -> ($EXT_IF) nat on $EXT_IF2 from <private_nets> to any -> ($EXT_IF2) # # Filter rules # pass all pass in from 192.168.100.0/24 to any rtable 0 pass in from 192.168.150.0/24 to any rtable 1
I woke up this morning on that the smartd was reporting an error: “smartd[1209]: Device: /dev/ad3, 1 Currently unreadable (pending) sectors” from one of the physical devices that was connected to my mirrored device. When I googled I found explanation:
Current count of unstable sectors (waiting for remapping). The raw value of this attribute indicates the total number of sectors waiting for remapping. Later, when some of these sectors are read successfully, the value is decreased. If errors still occur when reading some sector, the hard drive will try to restore the data, transfer it to the reserved disk area (spare area) and mark this sector as remapped. If this attribute value remains at zero, it indicates that the quality of the corresponding surface area is low.
smartctl -a /dev/ad3 showed the following:
... 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 1 198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 0 ...
So what needed to be done was to force the drive to write the area with new data and thus getting the sector either remapped or cleared from the error. As the disk was a mirrored drive and the other drive was working properly (+ I have backups in case of horror) I decided that I can remove the drive from the mirror using gmirror remove. Then I ran a long test with smartctl -t long /dev/ad3 and it reported no errors. After that I inserted the drive back to the mirror and as the drive was resynchronized S.M.A.R.T. Current_Pending_Sector value was decreased:
... 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 0 ...
The drive didn’t increase the Reallocated_Sector_Ct.
——
Update: Although I managed to correct the pending sectors couple of days after the first error occurred the drive started to fail totally. So tweaking and fixing was not very helpful as the errors started to escalate. In conclusion I decided to change the drive.
Smartd status just before I removed the drive:
smartd[1209]: Device: /dev/ad3, 348 Currently unreadable (pending) sectors
smartd[1209]: Device: /dev/ad3, 257 Offline uncorrectable sectors
Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 1 3 Spin_Up_Time 0x0027 239 238 021 Pre-fail Always - 1050 4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 54 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0 7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0 9 Power_On_Hours 0x0032 084 084 000 Old_age Always - 12022 10 Spin_Retry_Count 0x0032 100 253 000 Old_age Always - 0 11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 52 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 51 193 Load_Cycle_Count 0x0032 200 200 000 Old_age Always - 2 194 Temperature_Celsius 0x0022 124 107 000 Old_age Always - 23 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0 197 Current_Pending_Sector 0x0032 196 196 000 Old_age Always - 348 198 Offline_Uncorrectable 0x0030 197 197 000 Old_age Offline - 257 199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0 200 Multi_Zone_Error_Rate 0x0008 001 001 000 Old_age Offline - 24383
In the previous post I described how you can create virtual interfaces which can be used to fetch unique address for each interface using dhclient(8). To use these different addresses assigned to different virtual interfaces in your packet filter NAT you can use following pf.conf. It is important to note that the last filter policy needs to be floating as the packets will flow out from the physical interface and the return packets will arrive on the virtual interface.
# # Macros # INT_IF = "em0" EXT_IF = "bge0" VIRT_IF1 = "ngeth0" VIRT_IF2 = "ngeth1" VIRT_IF3 = "ngeth2" table <private_nets> persist { 127/8, 10/8, 172.16/12, 192.168/16, 169.254/16 } # # Options and default policy # set block-policy drop set state-policy if-bound # # Packet normalization # scrub in all scrub out on $EXT_IF all random-id scrub on $EXT_IF all reassemble tcp # # NAT/redirects # # NAT nat on $EXT_IF from <private_nets> to any -> \ { ($EXT_IF), ($VIRT_IF1), ($VIRT_IF2), ($VIRT_IF3) } # # Filter rules # block log all pass on $INT_IF all pass out all pass out on $EXT_IF from { ($VIRT_IF1), ($VIRT_IF2), ($VIRT_IF3) } to \ any keep state (floating)
With generic ethernet interface netgraph node it is possible to create new interfaces that are accessible with ifconfig(8) and with ethernet bridging netgraph node type it is possible to bridge between different ethernet node types. By combining these two types it is possible to create multiple interfaces with different ethernet (MAC) address on a same physical interface. Here is sample script that is modified from the netgraph ether.bridge example script that comes with standard FreeBSD distribution:
#!/bin/sh # # Author: Jan Melen-jan(at)melen.org # # This script sets up an Ethernet bridging network across multiple # ng_eiface(4) using the ng_bridge(4) and ng_ether(4) netgraph # node types. # BRIDGE_NAME="bnet0" VIRTUAL_IFACES="ngeth0 ngeth1 ngeth2" LOCAL_IFACE="bge0" # Routine to verify node's existence. vether_verify() { ngctl info ${BRIDGE_NAME}: >/dev/null 2>&1 if [ $? -ne 0 ]; then echo "${BRIDGE_NAME}: bridge network not found" exit 1 fi } # Start/restart routine. vether_start() { # Load netgraph KLD's as necessary. for KLD in ng_ether ng_bridge; do if ! kldstat -v | grep -qw ${KLD}; then echo -n "Loading ${KLD}.ko... " kldload ${KLD} || exit 1 echo "done" fi done # Reset all interfaces. vether_stop # Verify all interfaces exist. LINKNUM=2 ETHERADDR=`ifconfig ${LOCAL_IFACE} ether | grep "ether" | cut -b 8-22` for ETHER in ${VIRTUAL_IFACES}; do if ! ngctl info ${ETHER}: >/dev/null 2>&1; then ngctl mkpeer . eiface hook ether fi ifconfig ${ETHER} ether ${ETHERADDR}${LINKNUM} ifconfig ${ETHER} up || exit 1 LINKNUM=`expr ${LINKNUM} + 1` done # Create new ng_bridge(4) node, attached to the first interface. ngctl mkpeer ${LOCAL_IFACE}: bridge lower link0 || exit 1 ngctl name ${LOCAL_IFACE}:lower ${BRIDGE_NAME} || exit 1 ngctl connect ${LOCAL_IFACE}: ${BRIDGE_NAME}: upper link1 || exit 1 LINKNUM=2 # Attach other interfaces as well. for ETHER in ${VIRTUAL_IFACES}; do if [ ${LINKNUM} != 0 ]; then ngctl connect ${ETHER}: ${BRIDGE_NAME}: \ ether link${LINKNUM} || exit 1 fi LINKNUM=`expr ${LINKNUM} + 1` done # Set all interfaces in promiscuous mode and don't overwrite src addr. for ETHER in ${LOCAL_IFACE}; do ngctl msg ${ETHER}: setpromisc 1 ngctl msg ${ETHER}: setautosrc 0 done } # Stop routine. vether_stop() { ngctl kill ${BRIDGE_NAME}: >/dev/null 2>&1 for ETHER in ${VIRTUAL_IFACES} ${LOCAL_IFACE}; do ngctl kill ${ETHER}: >/dev/null 2>&1 done } # Main entry point. case $1 in start) vether_start ;; stop) vether_verify vether_stop ;; *) echo "usage: $0 [ start | stop ]" exit 1 esac