Amazon ec2 如何使用AmazonAMI在AmazonAWSEC2或EMR上安装GUI

我需要运行一个需要GUI界面来启动和配置的应用程序。我还需要能够在Amazon的EC2服务和EMR服务上运行这个应用程序。EMR要求意味着它必须在Amazon的LinuxAMI上运行

经过广泛的搜索，我一直无法找到任何现成的解决方案，特别是在亚马逊的AMI上运行的需求。最接近的匹配和最常引用的解决方案是。不幸的是，它是在RHEL6实例上开发的，该实例与Amazon的AMI差异很大，因此解决方案无法工作

---

我在下面发布我的解决方案。希望它能让其他人从花了数小时的实验中得到正确的配方。

下面是我在亚马逊的AMI上运行GUI的解决方案。我把它作为一个起点，但是为了让它在Amazon的AMI上工作，我不得不做很多修改。我还添加了额外的信息，以使这项工作以一种合理的自动化方式进行，这样一个需要多次使用此环境的个人就可以不费吹灰之力地完成

注：我在这篇文章中包括了很多评论。我提前道歉，但我认为如果有人需要做出修改，如果他们能够理解为什么会在修改过程中做出各种选择，这可能会对他们有所帮助

下面包含的脚本将在此过程中安装一些文件。有关这些脚本使用的文件和目录结构的列表，请参见第4节

第一步。安装桌面 在执行“yum update”之后，大多数解决方案都包含一行

sudo yum groupinstall -y "Desktop"

这一看似简单的步骤需要在AmazonAMI上付出更多的努力。此组未在AmazonAMI中配置（从现在起为AAMI）。默认情况下，AAMI安装并启用了亚马逊自己的存储库。还安装了epel repo，但默认情况下已禁用。启用epel后，我找到了桌面组，但没有用包填充它。我还发现了Xfce（另一个桌面替代品），它已经被填充。最终我决定安装Xfce而不是桌面。尽管如此，这并不是直截了当的，但最终还是找到了解决办法

这里值得注意的是，我尝试的第一件事是安装centos存储库并从那里安装桌面组。起初，这似乎很有希望。该组已完全填充了包。然而，经过一番努力，我最终决定，AAMI上已经安装的依赖项和包之间存在太多版本冲突

这让我从epel回购协议中选择了Xfce。由于epel repo已经安装在AAMI上，我认为与Amazon repo的依赖版本协调性会更好。这通常是正确的。在epel回购协议或亚马逊回购协议中发现了许多依赖关系。对于那些没有的，我可以在centos repo中找到它们，在大多数情况下，它们是叶依赖。因此，大部分问题来自centos repo中的少数依赖项，这些依赖项具有与amazon或epel repo冲突的子依赖项。最后需要一些黑客来绕过依赖冲突。我试着尽量减少这些。下面是安装Xfce的脚本

installGui.sh

#!/bin/bash

# echo each command
set -x

# assumes RSRC_DIR and IS_EMR set by parent script
YUM_RSRC_DIR=$RSRC_DIR/yum

sudo yum -y update

# Most info I've found on installing a GUI on AWS suggests to install using
#> sudo yum groupinstall -y "Desktop"
# This group is not available by default on the Amazon Linux AMI.  The group
# is listed if the epel repo is enabled, but it is empty.  I tried installing
# the centos repo, which does have support for this group, but it simply end
# up having to many dependency version conflicts with packages already installed
# by the Amazon repos.
#
# I found the path of least resistance to be installing the group Xfce from
# the epel repo. The epel repo is already included in amazon image, just not enabled.
# So I'm guessing there was at least some consideration by Amazon to align
# the dependency versions of this repo with the Amazon repos.
#
# My general approach to this problem was to start with the last command:
#> sudo yum groupinstall -y Xfce
# which will generate a list of missing dependencies.  The script below
# essentially works backwards through that list to eliminate all the
# missing dependencies.
#
# In general, many of the dependencies required by Xfce are found in either
# the epel repo or the Amazon repos.  Most of the remaining dependencies can be
# found in the centos repo, and either don't have any further dependencies, or if they
# do those dependencies are satisfied with the centos repo with no collisions
# in the epel or amazon repo.  Then there are a couple of oddball dependencies
# to clean up.

# if yum-config-manager is not found then install yum-utils
#> sudo yum install yum-utils
sudo yum-config-manager --enable epel

# install centos repo
# place the repo config @  /etc/yum.repos.d/centos.repo
sudo cp $YUM_RSRC_DIR/yum.repos.d/centos.repo /etc/yum.repos.d/

# The config centos.repo specifies the key with a URL.  If for some reason the key
# must be in a local file, it can be found here: https://www.centos.org/keys/RPM-GPG-KEY-CentOS-6
# It can be installed to the right location in one step:
#> wget -O /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 https://www.centos.org/keys/RPM-GPG-KEY-CentOS-6
# Note, a key file must also be installed in the system key ring.  The docs are a bit confusing
# on this, I found that I needed to run both gpg AND then followed by rpm, eg:
#> sudo gpg --import /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
#> sudo rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6

# I found there are a lot of version conflicts between the centos, Amazon and epel repos.
# So I did not enable the centos repo generally.  Instead I used the --enablerepo switch
# enable it explicitly for each yum command that required it.  This only works for yum.  If
# rpm must be used, then yum-config-manager must be used to enable/disable repos as a
# separate step.
#
# Another problem I ran into was yum installing the 32-bit (*.i686) package rather than
# the 64-bit (*.x86_64) verision of the package.  I never figured out why.  So I had
# to specify the *.x86_64 package explicitly.  The search tools (eg. 'whatprovides')
# did not list the 64 bit package either even though a manual search through the
# package showed the 64 bit components were present.
#
# Sometimes it is difficult to determine which package must be in installed to satisfy
# a particular dependency.  'whatprovides' is a very useful tool for this
#> yum --enablerepo centos whatprovides libgdk_pixbuf-2.0.so.0
#> rpm -q --whatprovides libgdk_pixbuf

sudo yum --enablerepo centos install -y gdk-pixbuf2.x86_64
sudo yum --enablerepo centos install -y gtk2.x86_64
sudo yum --enablerepo centos install -y libnotify.x86_64
sudo yum --enablerepo centos install -y gnome-icon-theme
sudo yum --enablerepo centos install -y redhat-menus
sudo yum --enablerepo centos install -y gstreamer-plugins-base.x86_64

# problem when we get to libvte, installing libvte requires expat, which conflicts with amazon lib
# the centos package version was older and did not install right lib version
# but … the expat dependency was coming from a dependency on python-libs.
# the easiest workaround was to install python using the amazon repo, that in turn
# installs a version of python libs that is compatible with the version of libexpat on the system.

sudo yum install -y python
sudo yum --enablerepo centos install -y vte.x86_64

sudo yum --enablerepo centos install -y libical.x86_64
sudo yum --enablerepo centos install -y gnome-keyring.x86_64

# another sticky point, xfdesktop requires desktop-backgrounds-basic, but ‘whatprovides’ does not 
# provide any packages for this query (not sure why).  It turns out this is provided by the centos 
# repo, installing ‘desktop-backgrounds-basic’ will try to install the package redhat-logos, but 
# unfortunately this is obsoleted by Amazon’s generic-logos package
# The only way I could find to get around this was to erase the generic logos package.
# This doesn't seem too risky since this is just images for the desktop and menus.
#
sudo yum erase -y generic-logos

# Amazon repo must be disabled to prevent interference with the install
# of redhat-logos
sudo yum --disablerepo amzn-main --enablerepo centos install -y redhat-logos

# next problem is a dependency on dbus.  The dependency comes from dbus-x11 in 
# centos repo.  It requires dbus version 1.2.24, the amazon image already has
# version 1.6.12 installed.  Since the dbus-x11 is only used by the GUI package,
# easiest way around this is to install dbus-x11 with no dependency checks.
# So it will use the newer version of dbus (should be OK).  The main thing that could be a problem
# here is if it skips some other dependency.  When doing manually, its possible to run the install until
# the only error left is the dbus dependency.  It’s a bit risky running in a script since, basically it’s assuming
# all the dependencies are already in place.
yumdownloader --enablerepo centos dbus-x11.x86_64
sudo rpm -ivh --nodeps dbus-x11-1.2.24-8.el6_6.x86_64.rpm
rm dbus-x11-1.2.24-8.el6_6.x86_64.rpm

sudo yum install -y xfdesktop.x86_64

# We need the version of poppler-glib from centos repo, but it is found in several repos.
# Disable the other repos for this step.
# On EMR systems a newer version of poppler is already installed.  So move up 1 level
# in dependency chain and force install of tumbler.

if [ $IS_EMR -eq 1 ]
then
    yumdownloader --enablerepo centos tumbler.x86_64
    sudo rpm -ivh --nodeps tumbler-0.1.21-1.el6.x86_64.rpm
else
    sudo yum --disablerepo amzn-main --disablerepo amzn-updates --disablerepo epel --enablerepo centos install -y poppler-glib
fi


sudo yum install  --enablerepo centos -y polkit-gnome.x86_64
sudo yum install  --enablerepo centos  -y control-center-filesystem.x86_64

sudo yum groupinstall -y Xfce

#!/bin/bash

# echo each command
set -x

BIN_DIR="${BASH_SOURCE%/*}"
ROOT_DIR=$(dirname $BIN_DIR)
RSRC_DIR=$ROOT_DIR/rsrc
VNC_DIR=$RSRC_DIR/vnc

# Install user config files into ec2-user home directory
# if it is available.  In practice, this should always
# be true

if [ -d "/home/ec2-user" ]
then
   USER_ACCT=ec2-user
else
   USER_ACCT=hadoop
fi

HOME_DIR="/home"

# Use existence of hadoop home directory as proxy to determine if
# this is an EMR system.  Can be used later to differentiate
# steps on EC2 system vs EMR.
if [ -d "/home/hadoop" ]
then
    IS_EMR=1
else
    IS_EMR=0
fi


# execute Xfce desktop install
. "$BIN_DIR/installXfce.sh"

# now roughly follow the following from step 3: https://devopscube.com/setup-gui-for-amazon-ec2-linux/

sudo yum install -y pixman pixman-devel libXfont

sudo yum -y install tigervnc-server


# install the user account configuration file.
# This setup assumes the user will always connect to the VNC server
# through an SSH tunnel.  This is generally more secure, easier to
# configure and easier to get correct than trying to allow direct
# connections via TCP.
# Therefore, config VNC server to only accept local connections, and
# no password required.
sudo cp $VNC_DIR/vncservers-$USER_ACCT /etc/sysconfig/vncservers

# install the user account, vnc config files

sudo mkdir $HOME_DIR/$USER_ACCT/.vnc
sudo chown $USER_ACCT:$USER_ACCT $HOME_DIR/$USER_ACCT/.vnc

# need xstartup file to tell vncserver to start the window manager
sudo cp $VNC_DIR/xstartup $HOME_DIR/$USER_ACCT/.vnc/
sudo chown $USER_ACCT:$USER_ACCT $HOME_DIR/$USER_ACCT/.vnc/xstartup

# Even though the VNC server is config'd to not require a passwd, the
# server still looks for the passwd file when it starts the session.
# It will fail if the passwd file is not found.
# The first time these scripts are run, the final step will fail.
# Then manually run
#> vncpasswd
# It will create the file ~/.vnc/passwd.  Then save this file to persistent
# storage so that it can be installed to the user account during
# server initialization.

sudo cp $ROOT_DIR/home/user/.vnc/passwd $HOME_DIR/$USER_ACCT/.vnc/
sudo chown $USER_ACCT:$USER_ACCT $HOME_DIR/$USER_ACCT/.vnc/passwd

# This script will be running as root if called from the EC2 launch
# command.  VNC server needs to be started as the user that
# you will connect to the server as (eg. ec2-user, hadoop, etc.)
sudo su -c "sudo service vncserver start" -s /bin/sh $USER_ACCT

# how to stop vncserver
# vncserver -kill :1

# On the remote client
# 1. start the ssh tunner
#> ssh -i ~/.ssh/<YOUR_KEY_FILE>.pem -L 5901:localhost:5901 -N ec2-user@<YOUR_SERVER_PUBLIC_IP>
#    for debugging connection use -vvv switch
# 2. connect to the vnc server using client on the remote machine.  When
#    prompted for the IP address, use 'localhost:5901'
#    This connects to port 5901 on your local machine, which is where the ssh
#    tunnel is listening.

以下是centos存储库配置文件的内容：

中央回购

[centos]
name=CentOS mirror
baseurl=http://repo1.ash.innoscale.net/centos/6/os/x86_64/
failovermethod=priority
enabled=0
gpgcheck=1
gpgkey=https://www.centos.org/keys/RPM-GPG-KEY-CentOS-6

如果您所需要的只是在AmazonAMI上安装一个桌面软件包，那么您就完成了。本文的其余部分将介绍如何配置VNC以通过SSH隧道访问桌面，以及如何打包所有这些内容，以便从脚本轻松启动实例

第二步。安装和配置VNC 下面是我安装GUI的顶层脚本。在配置了几个变量之后，它要做的第一件事就是调用上面步骤1中的脚本。这个脚本有一些额外的负担，因为我构建它是为了在常规ec2实例或emr上工作，并且作为root用户或ec2用户。基本步骤是

安装libXfont

安装tiger vnc服务器

安装VNC服务器配置文件

在用户主目录中创建.vnc目录

在.vnc目录中安装xstartup文件

在.vnc目录中安装一个伪passwd文件

启动VNC服务器

需要注意的几个关键点：

这假设您将通过SSH隧道访问VNC服务器。最终，这似乎是最简单、最可靠的安全方式。由于您的安全组规范中可能有一个SSH端口处于打开状态，因此不必对其进行任何更改。此外，VNC客户机/服务器的加密配置也不是直截了当的。似乎很容易犯错误，使您的通信未加密。此设置在vncservers文件中。-localhost开关告诉vnc仅接受本地连接。'-nolisten tcp'告诉关联的xserver模块也不要接受来自网络的连接。最后，'-SecurityTypes None'开关允许您在不键入passwd的情况下打开VNC会话，因为进入机器的唯一方法是通过ssh，因此额外的密码检查似乎是多余的

xstartup文件确定首次启动VNC会话时将启动什么。我注意到很多关于这个主题的帖子都跳过了这一点。如果您不告诉它启动Xfce桌面，那么在启动VNC时只会出现一个空白窗口。我这里的配置非常简单

尽管我在上面提到，VNC服务器配置为不提示输入密码，但它仍然需要.VNC目录中的passwd文件才能启动服务器。第一次运行脚本时，它将在尝试启动服务器时失败。通过ssh登录到计算机并运行“vncpasswd”。它将在.vnc目录中创建一个passwd文件，您可以将其保存为t的一部分

# The VNCSERVERS variable is a list of display:user pairs.
#
# Uncomment the lines below to start a VNC server on display :2
# as my 'myusername' (adjust this to your own).  You will also
# need to set a VNC password; run 'man vncpasswd' to see how
# to do that.  
#
# DO NOT RUN THIS SERVICE if your local area network is
# untrusted!  For a secure way of using VNC, see this URL:
# http://kbase.redhat.com/faq/docs/DOC-7028

# Use "-nolisten tcp" to prevent X connections to your VNC server via TCP.

# Use "-localhost" to prevent remote VNC clients connecting except when
# doing so through a secure tunnel.  See the "-via" option in the
# `man vncviewer' manual page.

# Use "-SecurityTypes None" to allow session login without a password.
# This should only be used in combination with "-localhost"
# Note: VNC server still looks for the passwd file in ~/.vnc directory
# when the session starts regardless of whether the user is
# required to enter a passwd.

# VNCSERVERS="2:myusername"
# VNCSERVERARGS[2]="-geometry 800x600 -nolisten tcp -localhost"
VNCSERVERS="1:ec2-user"
VNCSERVERARGS[1]="-geometry 1280x1024 -nolisten tcp -localhost -SecurityTypes None"

#!/bin/sh

unset SESSION_MANAGER
unset DBUS_SESSION_BUS_ADDRESS
# exec /etc/X11/xinit/xinitrc
/usr/share/vte/termcap/xterm &
/usr/bin/startxfce4 &

ssh -i ~/.ssh/<YOUR_KEY_FILE>.pem -L 5901:localhost:5901 -N ec2-user@<YOUR_SERVER_PUBLIC_IP>

sudo mkdir /mnt/YOUR_MOUNT_POINT_DIR
sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 fs-YOUR_EFS_ID.efs.us-east-1.amazonaws.com:/ /mnt/YOUR_MOUNT_POINT_DIR

aws ec2 run-instances --user-data file:///PATH_TO_YOUR_SCRIPT/top.sh  ... other options

aws emr create-cluster --bootstrap-actions '[{"Path":"s3://YOUR_BUCKET/YOUR_DIR/top.sh","Name":"Custom action"}]' ... other options

#!/bin/bash

# note: conditional bootstrap function run-if has a bug, workaround ...
# this function adapted from https://forums.aws.amazon.com/thread.jspa?threadID=222418
# Determine if we are running on the master node.
# 0 - running on master, or non EMR node
# 1 - running on a task or core node

check_if_master_or_non_emr() {
    python - <<'__SCRIPT__'
import sys
import json

instance_file = "/mnt/var/lib/info/instance.json"

try:
    with open(instance_file) as f:
        props = json.load(f)
    is_master_or_non_emr = props.get('isMaster', False)

except IOError as ex:
    is_master_or_non_emr = True   # file will not exist when testing on a non-emr machine

if is_master_or_non_emr:
    sys.exit(1)
else:
    sys.exit(0)
__SCRIPT__
}

check_if_master_or_non_emr
IS_MASTER_OR_NON_EMR=$?

# If this machine is part of EMR cluster, then ONLY install on the MASTER node

if [ $IS_MASTER_OR_NON_EMR -eq 1 ]
then
    sudo mkdir /mnt/YOUR_MOUNT_POINT_DIR

    sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 fs-YOUR_EFS_ID.efs.us-east-1.amazonaws.com:/ /mnt/YOUR_MOUNT_POINT_DIR

    . /mnt/YOUR_MOUNT_POINT_DIR/bin/installGui.sh
fi

exit 0