User:Tanner: Difference between revisions

From Maemo Wiki
Jump to navigationJump to search
imported>tanner
m chroot
imported>tanner
domain change maemory.com
 
(29 intermediate revisions by 4 users not shown)
Line 26: Line 26:
To replace several tools in scratchbox you can also use the following line as a template
To replace several tools in scratchbox you can also use the following line as a template
  SBOX_REDIRECT_IGNORE:=$(shell echo /usr/bin/{perl,dh_*} | sed "s/ /:/g")
  SBOX_REDIRECT_IGNORE:=$(shell echo /usr/bin/{perl,dh_*} | sed "s/ /:/g")
== CLI icon ==
For your convenience I have uuencoded Tim Samoff's [http://lists.maemo.org/pipermail/maemo-community/2010-January/003870.html CLI icon]
which is supposed to be added to the debian/control file of CLI apps. You can download it [http://n900.tannerlab.com/tools/cli.icon.txt here].
== [http://maemo.org/packages/view/kernel-flasher-maemo/ custom Linux kernel] ==
My custom kernel which contains probably everything one could need on the N900:
This custom kernel contains additional modules for IPv6, packet filtering,
NAT, tunneling, kernel configuration, Wifi mesh networking,
ext4, XFS, reiserfs, NTFS reading, NILFS2, ISO9660, UDF, CIFS, automounter,
UNIONFS, device mapper and dmloop, cryptography, cryptoloop,
EFI partitions, UTF8 codepages, mouse+joystick input,
USB/IP and generic USB device drivers.
It shows a boot logo.
Just install the package "kernel-flasher-maemo" from extras-devel/non-free (section System) and reboot.
Reverting to the original kernel is simple:  "apt-get install --reinstall kernel kernel-flasher"  or reflash only the original kernel image.
see [http://talk.maemo.org/showthread.php?t=43420 TMO page] for more details.
== GNU tools ==
the crippled busybox is installed on the N900.
command line users prefer the real GNU tools.
these are the optified packages for the device.
Most of them do not take use more RAM than busybox (a few KBs)!
[http://maemo.org/packages/view/coreutils-gnu/ coreutils-gnu]
[http://maemo.org/packages/view/diffutils-gnu/ diffutils-gnu]
[http://maemo.org/packages/view/findutils-gnu/ findutils-gnu]
[http://maemo.org/packages/view/grep-gnu/ grep-gnu]
[http://maemo.org/packages/view/sed-gnu/ sed-gnu]
[http://maemo.org/packages/view/tar-gnu/ tar-gnu]


== [http://maemo.org/packages/view/texlive-latex-base/ TeXLive 2009] ==
== [http://maemo.org/packages/view/texlive-latex-base/ TeXLive 2009] ==
Line 48: Line 81:
== [http://maemo.org/packages/view/tcl8.5/ tcl8.5] [http://maemo.org/packages/view/tk8.5/ tk8.5] [http://maemo.org/packages/view/python-tk/ python-tk] ==
== [http://maemo.org/packages/view/tcl8.5/ tcl8.5] [http://maemo.org/packages/view/tk8.5/ tk8.5] [http://maemo.org/packages/view/python-tk/ python-tk] ==


== [http://maemo.org/packages/view/lzma/ lzma] ==
== [http://maemo.org/packages/view/lzma/ lzma] and its successor [http://maemo.org/packages/view/xz-utils/ xz-utils] ==


an extremely efficient compressor. it is much faster than bzip2 for decompression and yields about 30% smaller files.
an extremely efficient compressor. it is much faster than bzip2 for decompression and yields about 30% smaller files.
Line 61: Line 94:
some experimental ports which I have not yet uploaded to extras-devel can be found in my repository
some experimental ports which I have not yet uploaded to extras-devel can be found in my repository


  deb http://www.maemory.com/N900/ fremantle free incompatible
  deb http://n900.tannerlab.com/ fremantle free incompatible


currently: python2.6, perl5.10, GNU compiler collection 4.4.3, autoconf 2.64, automake1.10
currently: python2.6, perl5.10, GNU compiler collection 4.4.3, autoconf 2.64, automake1.10
Line 68: Line 101:


= Ideas/plans: =
= Ideas/plans: =
== [https://garage.maemo.org/projects/moebian/ Moebian]: Maemo/MeeGo on Debian ==
[http://wiki.maemo.org/Maemo_on_Debian Wiki] and [http://talk.maemo.org/showthread.php?t=44967 discussion]
== Brainstorm: [https://maemo.org/community/brainstorm/view/more_efficient_and_flexible_use_of_internal_flash Repartitioning] ==
[http://talk.maemo.org/showthread.php?t=37869 brainstorm discussion]
[http://talk.maemo.org/showthread.php?t=35122 repartitioning thread]
== Brainstorm: [http://maemo.org/community/brainstorm/view/stylus_and_finger_friendly_uis/ Stylus and finger friendly UIs] ==
[http://talk.maemo.org/showthread.php?p=525854 brainstorm discussion]


== alternative USB host solution ==
== alternative USB host solution ==
Line 84: Line 130:
and the search paths of Maemo need to be adjusted in /etc/profile and  /etc/osso-af-init/af-defines.sh:
and the search paths of Maemo need to be adjusted in /etc/profile and  /etc/osso-af-init/af-defines.sh:


* PATH=$PATH:/opt/maemo/usr/bin  (and sbin for root)
* PATH=$PATH:/opt/maemo/bin  (and sbin for root)
* ld.so.conf: add /opt/maemo/usr/lib
* ld.so.conf: add /opt/maemo/lib
* locales: [https://bugs.maemo.org/show_bug.cgi?id=8363 LOCPATH]=LOCPATH=/usr/share/locale:/opt/maemo/usr/share/locale
* locales: [https://bugs.maemo.org/show_bug.cgi?id=8363 LOCPATH]=LOCPATH=/usr/share/locale:/opt/maemo/share/locale
* MANPATH=/opt/maemo/usr/share/man
* MANPATH=/opt/maemo/share/man
* pkgconfig: /usr/share/pkgconfig:/opt/maemo/usr/share/pkgconfig
* pkgconfig: /usr/share/pkgconfig:/opt/maemo/share/pkgconfig
* info in info?: /usr/share/info:/opt/maemo/usr/share/info
* info in info?: /usr/share/info:/opt/maemo/share/info
* apps in hildon ?: /usr/share/applications:/opt/maemo/usr/share/applications
* apps in hildon: [https://bugs.maemo.org/show_bug.cgi?id=10531 Bug about Application Menu] /usr/share/applications:/opt/maemo/share/applications
* icons in hildon ?: /usr/share/icons:/opt/maemo/usr/share/icons
* icons in hildon ?: /usr/share/icons:/opt/maemo/share/icons
this list may be incomplete - please help to complete it
this list may be incomplete - please help to complete it


Line 100: Line 146:
TMPDIR=/var/tmp should be moved to /opt/tmp
TMPDIR=/var/tmp should be moved to /opt/tmp
or /var/opt should symlink to /opt/var and contain a larger tmp directory.
or /var/opt should symlink to /opt/var and contain a larger tmp directory.
A package for Maemo (tested on N800) that allows a package to set environment variables like XDG_DATA_DIRS is available: [http://svn.nomike.org/playground/trunk/opt-support/ opt-support]. I propose packages depend on opt-support and in their postinst add their variables to the file "/etc/profile.env". -- dannym
== chroot instead of /opt ==
== chroot instead of /opt ==
I have described my currently preferred [http://maemo.org/community/brainstorm/view/remove_256mb_limitation_of_the_rootfs_partition_in_the_n900/ solution as #5]
I have described my currently preferred [http://maemo.org/community/brainstorm/view/remove_256mb_limitation_of_the_rootfs_partition_in_the_n900/ solution as #5]
Line 110: Line 159:
[http://maemo.org/packages/repository/latest/fremantle_extras-devel_free_armel/all/ latest packages]
[http://maemo.org/packages/repository/latest/fremantle_extras-devel_free_armel/all/ latest packages]


== Repartitioning ==
[https://garage.maemo.org/builder/fremantle/?C=M;O=D autobuilder current builds]


[https://maemo.org/community/brainstorm/view/more_efficient_and_flexible_use_of_internal_flash brainstorm]
== [[Repartitioning the flash]] ==
[http://talk.maemo.org/showthread.php?t=37869 brainstorm discussion]
[http://talk.maemo.org/showthread.php?t=35122 repartitioning thread]


[http://talk.maemo.org/showpost.php?p=424811&postcount=66 how to swap FAT and ext3 parition]
[http://talk.maemo.org/showpost.php?p=424811&postcount=66 how to swap FAT and ext3 parition]
[http://talk.maemo.org/showpost.php?p=425359&postcount=68 how to implement USB mass storage with a loopback file]
[http://talk.maemo.org/showpost.php?p=425359&postcount=68 how to implement USB mass storage with a loopback file]


== Setting up a repository mirror ==
== Setting up a repository mirror on your server ==
 
[http://n900.tannerlab.com/mirror/repro-maemo.tgz reprepro configuration]
 
== Safely free some space on rootfs ==
 
[http://n900.tannerlab.com/tools/moveroot.sh This script] shows you how much space is freed.
It can also detect whether some directories have already been moved.
WARNING: Although this script works for most users, you should always perform a backup before running it.
Do not use this script if you have already used some other methods or scripts from [[Free_up_rootfs_space]]!
In this case reflashing is recommended.
 
# install the rootsh package from extras
# download the [http://n900.tannerlab.com/tools/moveroot.sh script] to your MyDocs (or upload it via SSH)
# in Terminal run
sudo gainroot
sh /home/user/MyDocs/moveroot.sh
 
It should have freed about 95MB on your rootfs.
Upgrading the firmware OTA after using the script is supposed to work.
If you reflash the firmware the script will automatically remove the files from the old firmware
when you run it again.


[http://www.maemory.com/mirror/repro-maemo.tgz reprepro configuration]
[http://n900.tannerlab.com/tools/unmoveroot.sh This untested script] reverts the modifications and moves as many as possible files back to rootfs.
Make sure that you have enough space on rootfs for that. Otherwise it will abort the operation. Instructions are the same as for moveroot.sh,
except that you call unmoveroot.sh.
 
== On-device flashing of a custom kernel image ==
 
This how to flash a kernel on-device (without USB/flasher 3.5):
 
# you have a kernel image /path/zImage (with it's modules installed)
# install the package "sdk-fiasco-gen" from SDK
# become root: "sudo gainroot"
# make a "fiasco" :)
fiasco-gen -o /path/zImage.fiasco -g -k /path/zImage
# flash it
/sbin/fiasco-image-update /path/zImage.fiasco


== Reflashing & Restore ==
== Reflashing & Restore ==
Line 130: Line 212:
# store all settings with Backup on my SD card
# store all settings with Backup on my SD card
# reflash with standard http://wiki.maemo.org/Updating_the_firmware, i.e.
# reflash with standard http://wiki.maemo.org/Updating_the_firmware, i.e.
  sudo flasher-3.5 -F RX-51_2009SE_2.2009.51-1_PR_COMBINED_MR0_ARM.bin -f -R
  sudo flasher-3.5 -F ''<current firmware image>''.bin -f -R
# restore old backup except programs
# restore old backup except programs
# install gainroot
# install rootsh from System
# in Terminal:
# in Terminal:
  sudo su -
  sudo sh /home/backup/restore.sh
/home/backup/restore
with [http://n900.tannerlab.com/tools/restore.sh this script (for PR<1.2] or  [http://n900.tannerlab.com/tools/restore1.2.sh this script (PR1.2)] in /home/backup (feedback and patches are welcome).
with following script in /home/backup (feedback and patches are welcome).
It expects that you made a backup of /root and your selected modified files (e.g. /etc/ssh) in /home/backup.
It expects that you made a backup of /root and your selected modified files (e.g. /etc/ssh) in /home/backup.
The file "pkgs" should contain a list (line by line) of the package names you want to install.
The file "pkgs" should contain a list (line by line) of the package names you want to install.
#!/bin/sh
# N900 restore script
# © 2010 by Thomas Tanner <tanner@maemory.com>
# licensed under GPLv3
set -x
# script and data must be in /home/backup
cd /home/backup
# remove old opt
rm -rf /home/opt
mkdir /home/opt
# copy /root backup
rm -rf /root
cp -a root /
# copy modifed files for /etc
cp -a etc/* /etc
# enable user ssh login with ~/.ssh/authorized_keys
sed 's/^user:!/user:\*/' /etc/passwd > /etc/passwd.tmp
cp /etc/passwd.tmp /etc/passwd # mv
# move root stuff to /home
dirs="usr/share/icons usr/share/nokia-maps usr/share/themes usr/share/fonts usr/share/locale usr/lib/locale var/lib/apt"
#optional: var/lib/dpkg
if test -d /home/var/cache/apt; then
  # keep existing apt cache
  rm -rf /var/cache/apt
  ln -s /home/var/cache/apt /var/cache/apt
else
  dirs="$dirs var/cache/apt"
fi
(cd / && du -sc $dirs)
for d in $dirs; do
    test -L /$d && continue
    echo moving /$d
    rm -rf /home/$d
    mkdir -p /home/$d
    cp -a /$d /home/$d/..
    rm -rf /$d
    ln -s /home/$d /$d
done
mount /home/user/MyDocs # non-standard partition layout
apt-get update
apt-get -y install mc python-support openssh-server
# install my standard set of packages
apt-get -y install `cat pkgs`

Latest revision as of 06:50, 21 June 2010

Packages

info about some of the packages I've uploaded to extras-devel

see also http://wiki.maemo.org/Packaging

A backport of Debhelper 7 for Fremantle is available in extras-devel. It works transparently and can coexist with debhelper 5 in the SDK. It also works on the autobuilder, if you specify

Build-Depends: debhelper7

in debian/control. The following lines in debian/rules are necessary to use it:

PATH:=/usr/bin/dh7:/usr/bin:$(PATH)
export PATH
SBOX_REDIRECT_IGNORE=/usr/bin/perl
export SBOX_REDIRECT_IGNORE

We are working on enhancement to make optification is easier, i.e. direct installation of files in the /opt hierarchy.

If you also need a more recent cdbs, then use the package cdbs-dh7, which conflicts with the standard cdbs and does not work on autobuilder yet.

To replace several tools in scratchbox you can also use the following line as a template

SBOX_REDIRECT_IGNORE:=$(shell echo /usr/bin/{perl,dh_*} | sed "s/ /:/g")

CLI icon

For your convenience I have uuencoded Tim Samoff's CLI icon which is supposed to be added to the debian/control file of CLI apps. You can download it here.

My custom kernel which contains probably everything one could need on the N900:

This custom kernel contains additional modules for IPv6, packet filtering,
NAT, tunneling, kernel configuration, Wifi mesh networking,
ext4, XFS, reiserfs, NTFS reading, NILFS2, ISO9660, UDF, CIFS, automounter,
UNIONFS, device mapper and dmloop, cryptography, cryptoloop,
EFI partitions, UTF8 codepages, mouse+joystick input,
USB/IP and generic USB device drivers.
It shows a boot logo.

Just install the package "kernel-flasher-maemo" from extras-devel/non-free (section System) and reboot. Reverting to the original kernel is simple: "apt-get install --reinstall kernel kernel-flasher" or reflash only the original kernel image. see TMO page for more details.

GNU tools

the crippled busybox is installed on the N900. command line users prefer the real GNU tools. these are the optified packages for the device. Most of them do not take use more RAM than busybox (a few KBs)! coreutils-gnu diffutils-gnu findutils-gnu grep-gnu sed-gnu tar-gnu

talk page

R (statistics)

bug: installation fails with ucfr missing solution: wait or edit r-base-core.postinst and comment line 44 out

bug: linking against BLAS and LAPACK currently hardcodes the links found in the -dev package.

some discussion

bonnie++ (filesystem benchmarks)

see post and ff

lzma and its successor xz-utils

an extremely efficient compressor. it is much faster than bzip2 for decompression and yields about 30% smaller files.

bug: keyboard doesn't work. bug: window close button is not visible

experimental packages

some experimental ports which I have not yet uploaded to extras-devel can be found in my repository

deb http://n900.tannerlab.com/ fremantle free incompatible

currently: python2.6, perl5.10, GNU compiler collection 4.4.3, autoconf 2.64, automake1.10


Ideas/plans:

Moebian: Maemo/MeeGo on Debian

Wiki and discussion

Brainstorm: Repartitioning

brainstorm discussion repartitioning thread

brainstorm discussion

alternative USB host solution

USB/IP

Ideally addon software should be installed only in /opt without any symlinks in / (NAND) in accordance with FHS. /usr/local should be moved and symlinked to /opt/local and is reserved for own programs not maintained by dpkg.

A package may either install its files directly (for small packages) in the /opt/maemo hierarchy (bin,etc,lib,man,share) or in a separate directory /opt/<package> with its own hierarchy (bin,etc,lib,man,share). For the latter solution packages should symlink all non-private files to /opt/maemo/{bin,etc,lib,man,share} and the search paths of Maemo need to be adjusted in /etc/profile and /etc/osso-af-init/af-defines.sh:

  • PATH=$PATH:/opt/maemo/bin (and sbin for root)
  • ld.so.conf: add /opt/maemo/lib
  • locales: LOCPATH=LOCPATH=/usr/share/locale:/opt/maemo/share/locale
  • MANPATH=/opt/maemo/share/man
  • pkgconfig: /usr/share/pkgconfig:/opt/maemo/share/pkgconfig
  • info in info?: /usr/share/info:/opt/maemo/share/info
  • apps in hildon: Bug about Application Menu /usr/share/applications:/opt/maemo/share/applications
  • icons in hildon ?: /usr/share/icons:/opt/maemo/share/icons

this list may be incomplete - please help to complete it

As NAND is much faster than eMMC a cache manager could mantain a list of very often used binaries and libraries in /opt and automatically copy them and symlink them to a fixed size cache in NAND.

TMPDIR=/var/tmp should be moved to /opt/tmp or /var/opt should symlink to /opt/var and contain a larger tmp directory.

A package for Maemo (tested on N800) that allows a package to set environment variables like XDG_DATA_DIRS is available: opt-support. I propose packages depend on opt-support and in their postinst add their variables to the file "/etc/profile.env". -- dannym

chroot instead of /opt

I have described my currently preferred solution as #5


Tips and tricks:

fresh builds

latest packages

autobuilder current builds

how to swap FAT and ext3 parition how to implement USB mass storage with a loopback file

Setting up a repository mirror on your server

reprepro configuration

Safely free some space on rootfs

This script shows you how much space is freed. It can also detect whether some directories have already been moved. WARNING: Although this script works for most users, you should always perform a backup before running it. Do not use this script if you have already used some other methods or scripts from Free_up_rootfs_space! In this case reflashing is recommended.

  1. install the rootsh package from extras
  2. download the script to your MyDocs (or upload it via SSH)
  3. in Terminal run
sudo gainroot
sh /home/user/MyDocs/moveroot.sh

It should have freed about 95MB on your rootfs. Upgrading the firmware OTA after using the script is supposed to work. If you reflash the firmware the script will automatically remove the files from the old firmware when you run it again.

This untested script reverts the modifications and moves as many as possible files back to rootfs. Make sure that you have enough space on rootfs for that. Otherwise it will abort the operation. Instructions are the same as for moveroot.sh, except that you call unmoveroot.sh.

On-device flashing of a custom kernel image

This how to flash a kernel on-device (without USB/flasher 3.5):

  1. you have a kernel image /path/zImage (with it's modules installed)
  2. install the package "sdk-fiasco-gen" from SDK
  3. become root: "sudo gainroot"
  4. make a "fiasco" :)
fiasco-gen -o /path/zImage.fiasco -g -k /path/zImage
  1. flash it
/sbin/fiasco-image-update /path/zImage.fiasco 

Reflashing & Restore

this is a slightly modified version of the script I use to restore my N900 after a reflash. My reflashing procedure is:

  1. store all settings with Backup on my SD card
  2. reflash with standard http://wiki.maemo.org/Updating_the_firmware, i.e.
sudo flasher-3.5 -F <current firmware image>.bin -f -R
  1. restore old backup except programs
  2. install rootsh from System
  3. in Terminal:
sudo sh /home/backup/restore.sh

with this script (for PR<1.2 or this script (PR1.2) in /home/backup (feedback and patches are welcome). It expects that you made a backup of /root and your selected modified files (e.g. /etc/ssh) in /home/backup. The file "pkgs" should contain a list (line by line) of the package names you want to install.