深度学习主机环境配置: Ubuntu16.04+Nvidia GTX 1080+CUDA8.0

接上文《深度学习主机攒机小记》,这台GTX1080主机准备好之后,就是配置深度学习环境了,这里选择了比较熟悉Ubuntu系统,不过是最新的16.04版本,另外在Nvidia GTX1080的基础上安装相关GPU驱动,外加CUDA8.0,因为都比较新,所以踩了很多坑。

1. 安装Ubuntu16.04

不考虑双系统,直接安装 Ubuntu16.04,从ubuntu官方下载64位版本: ubuntu-16.04-desktop-amd64.iso 。

在MAC下制作了 Ubuntu USB 安装盘,具体方法可参考: 在MAC下使用ISO制作Linux的安装USB盘,之后通过Bios引导U盘启动安装Ubuntu系统:

1)一开始安装就踩了一个坑,选择”Install Ubuntu”回车后过一会儿屏幕显示“输入不支持”,google了好多方案,最终和ubuntu对显卡的支持有关,需要手动添加显卡选项: nomodeset,使其支持Nvidia系列显卡,参考:安装ubuntu黑屏问题的解决 or How do I set ‘nomodeset’ after I’ve already installed Ubuntu?

2) 磁盘分区,全部干掉之前主机自带的Window 10系统,分区为 /boot, /, /home 等几个目录,同时把第二块4T硬盘也挂载了上去,作为数据盘。

3)安装完毕后Ubuntu 16.04的分辨率很低,在显卡驱动未安装之前,可以手动修改一下grub文件:

sudo vim /etc/default/grub

# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command `vbeinfo’
#GRUB_GFXMODE=640×480
# 这里分辨率自行设置
GRUB_GFXMODE=1024×768

sudo update-grub

4)安装SSH Server,这样可以远程ssh访问这台GTX1080主机:

sudo apt-get install openssh-server

5)更新Ubuntu16.04源,用的是中科大的源

cd /etc/apt/
sudo cp sources.list sources.list.bak
sudo vi sources.list

把下面的这些源添加到source.list文件头部:

deb http://mirrors.ustc.edu.cn/ubuntu/ xenial main restricted universe multiverse
deb http://mirrors.ustc.edu.cn/ubuntu/ xenial-security main restricted universe multiverse
deb http://mirrors.ustc.edu.cn/ubuntu/ xenial-updates main restricted universe multiverse
deb http://mirrors.ustc.edu.cn/ubuntu/ xenial-proposed main restricted universe multiverse
deb http://mirrors.ustc.edu.cn/ubuntu/ xenial-backports main restricted universe multiverse
deb-src http://mirrors.ustc.edu.cn/ubuntu/ xenial main restricted universe multiverse
deb-src http://mirrors.ustc.edu.cn/ubuntu/ xenial-security main restricted universe multiverse
deb-src http://mirrors.ustc.edu.cn/ubuntu/ xenial-updates main restricted universe multiverse
deb-src http://mirrors.ustc.edu.cn/ubuntu/ xenial-proposed main restricted universe multiverse
deb-src http://mirrors.ustc.edu.cn/ubuntu/ xenial-backports main restricted universe multiverse

最后更新源和更新已安装的包:

sudo apt-get update
sudo apt-get upgrade

2. 安装GTX1080驱动

安装 Nvidia 驱动 367.27

sudo add-apt-repository ppa:graphics-drivers/ppa

第一次运行出现如下的警告:

Fresh drivers from upstream, currently shipping Nvidia.

## Current Status

We currently recommend: `nvidia-361`, Nvidia’s current long lived branch.
For GeForce 8 and 9 series GPUs use `nvidia-340`
For GeForce 6 and 7 series GPUs use `nvidia-304`

## What we’re working on right now:

– Normal driver updates
– Investigating how to bring this goodness to distro on a cadence.

## WARNINGS:

This PPA is currently in testing, you should be experienced with packaging before you dive in here. Give us a few days to sort out the kinks.

Volunteers welcome! See also: https://github.com/mamarley/nvidia-graphics-drivers/

http://www.ubuntu.com/download/desktop/contribute
更多信息: https://launchpad.net/~graphics-drivers/+archive/ubuntu/ppa
按回车继续或者 Ctrl+c 取消添加

回车后继续:

sudo apt-get update
sudo apt-get install nvidia-367
sudo apt-get install mesa-common-dev
sudo apt-get install freeglut3-dev

之后重启系统让GTX1080显卡驱动生效。

3. 下载和安装CUDA

在安装CUDA之前,google了一下,发现在Ubuntu16.04下安装CUDA7.5问题多多,幸好CUDA8已出,支持GTX1080:

New in CUDA 8

Pascal Architecture Support
Out of box performance improvements on Tesla P100, supports GeForce GTX 1080
Simplify programming using Unified memory on Pascal including support for large datasets, concurrent data access and atomics*
Optimize Unified Memory performance using new data migration APIs*
Faster Deep Learning using optimized cuBLAS routines for native FP16 computation
Developer Tools
Quickly identify latent system-level bottlenecks using the new critical path analysis feature
Improve productivity with up to 2x faster NVCC compilation speed
Tune OpenACC applications and overall host code using new profiling extensions
Libraries
Accelerate graph analytics algorithms with nvGRAPH
New cuBLAS matrix multiply optimizations for matrices with sizes smaller than 512 and for batched operation

不过下载CUDA需要注册和登陆NVIDIA开发者账号,CUDA8下载页面提供了很详细的系统选择和安装说明,

这里选择了Ubuntu16.04系统runfile安装方案,千万不要选择deb方案,前方无数坑:

屏幕快照 2016-07-15 上午8.25.37

下载的“cuda_8.0.27_linux.run”有1.4G,按照Nivdia官方给出的方法安装CUDA8:

sudo sh cuda_8.0.27_linux.run --tmpdir=/opt/temp/

这里加了–tmpdir主要是直接运行“sudo sh cuda_8.0.27_linux.run”会提示空间不足的错误,其实是全新的电脑主机,硬盘足够大的,google了以下发现加个tmpdir就可以了:

Not enough space on parition mounted at /.
Need 5091561472 bytes.

Disk space check has failed. Installation cannot continue.

执行后会有一系列提示让你确认,非常非常非常非常关键的地方是是否安装361这个低版本的驱动:

Install NVIDIA Accelerated Graphics Driver for Linux-x86_64 361.62?

答案必须是n,否则之前安装的GTX1080驱动就白费了,而且问题多多。

Logging to /opt/temp//cuda_install_6583.log
Using more to view the EULA.
End User License Agreement
————————–

Preface
——-

The following contains specific license terms and conditions
for four separate NVIDIA products. By accepting this
agreement, you agree to comply with all the terms and
conditions applicable to the specific product(s) included
herein.

Do you accept the previously read EULA?
accept/decline/quit: accept

Install NVIDIA Accelerated Graphics Driver for Linux-x86_64 361.62?
(y)es/(n)o/(q)uit: n

Install the CUDA 8.0 Toolkit?
(y)es/(n)o/(q)uit: y

Enter Toolkit Location
[ default is /usr/local/cuda-8.0 ]:

Do you want to install a symbolic link at /usr/local/cuda?
(y)es/(n)o/(q)uit: y

Install the CUDA 8.0 Samples?
(y)es/(n)o/(q)uit: y

Enter CUDA Samples Location
[ default is /home/textminer ]:

Installing the CUDA Toolkit in /usr/local/cuda-8.0 …
Installing the CUDA Samples in /home/textminer …
Copying samples to /home/textminer/NVIDIA_CUDA-8.0_Samples now…
Finished copying samples.

===========
= Summary =
===========

Driver: Not Selected
Toolkit: Installed in /usr/local/cuda-8.0
Samples: Installed in /home/textminer

Please make sure that
– PATH includes /usr/local/cuda-8.0/bin
– LD_LIBRARY_PATH includes /usr/local/cuda-8.0/lib64, or, add /usr/local/cuda-8.0/lib64 to /etc/ld.so.conf and run ldconfig as root

To uninstall the CUDA Toolkit, run the uninstall script in /usr/local/cuda-8.0/bin

Please see CUDA_Installation_Guide_Linux.pdf in /usr/local/cuda-8.0/doc/pdf for detailed information on setting up CUDA.

***WARNING: Incomplete installation! This installation did not install the CUDA Driver. A driver of version at least 361.00 is required for CUDA 8.0 functionality to work.
To install the driver using this installer, run the following command, replacing with the name of this run file:
sudo .run -silent -driver

Logfile is /opt/temp//cuda_install_6583.log

安装完毕后,再声明一下环境变量,并将其写入到 ~/.bashrc 的尾部:

export PATH=/usr/local/cuda-8.0/bin\${PATH:+:\${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64\${LD_LIBRARY_PATH:+:\${LD_LIBRARY_PATH}}

最后再来测试一下CUDA,运行:

nvidia-smi

结果如下所示:

屏幕快照 2016-07-15 上午8.44.43

再来试几个CUDA例子:

cd 1_Utilities/deviceQuery
make

这里如果提示gcc版本过高,可以安装低版本的gcc并做软连接替换,具体方法请自行google,我用低版本的gcc4.9替换了ubuntu16.04自带的gcc5.x版本。

“/usr/local/cuda-8.0″/bin/nvcc -ccbin g++ -I../../common/inc -m64 -gencode arch=compute_20,code=sm_20 -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_37,code=sm_37 -gencode arch=compute_50,code=sm_50 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_60,code=compute_60 -o deviceQuery.o -c deviceQuery.cpp
“/usr/local/cuda-8.0″/bin/nvcc -ccbin g++ -m64 -gencode arch=compute_20,code=sm_20 -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_37,code=sm_37 -gencode arch=compute_50,code=sm_50 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_60,code=compute_60 -o deviceQuery deviceQuery.o
mkdir -p ../../bin/x86_64/linux/release
cp deviceQuery ../../bin/x86_64/linux/release

执行 ./deviceQuery ,得到:

./deviceQuery Starting…

CUDA Device Query (Runtime API) version (CUDART static linking)

Detected 1 CUDA Capable device(s)

Device 0: “GeForce GTX 1080”
CUDA Driver Version / Runtime Version 8.0 / 8.0
CUDA Capability Major/Minor version number: 6.1
Total amount of global memory: 8112 MBytes (8506179584 bytes)
(20) Multiprocessors, (128) CUDA Cores/MP: 2560 CUDA Cores
GPU Max Clock rate: 1835 MHz (1.84 GHz)
Memory Clock rate: 5005 Mhz
Memory Bus Width: 256-bit
L2 Cache Size: 2097152 bytes
Maximum Texture Dimension Size (x,y,z) 1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)
Maximum Layered 1D Texture Size, (num) layers 1D=(32768), 2048 layers
Maximum Layered 2D Texture Size, (num) layers 2D=(32768, 32768), 2048 layers
Total amount of constant memory: 65536 bytes
Total amount of shared memory per block: 49152 bytes
Total number of registers available per block: 65536
Warp size: 32
Maximum number of threads per multiprocessor: 2048
Maximum number of threads per block: 1024
Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
Max dimension size of a grid size (x,y,z): (2147483647, 65535, 65535)
Maximum memory pitch: 2147483647 bytes
Texture alignment: 512 bytes
Concurrent copy and kernel execution: Yes with 2 copy engine(s)
Run time limit on kernels: Yes
Integrated GPU sharing Host Memory: No
Support host page-locked memory mapping: Yes
Alignment requirement for Surfaces: Yes
Device has ECC support: Disabled
Device supports Unified Addressing (UVA): Yes
Device PCI Domain ID / Bus ID / location ID: 0 / 1 / 0
Compute Mode:
< Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >

deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 8.0, CUDA Runtime Version = 8.0, NumDevs = 1, Device0 = GeForce GTX 1080
Result = PASS

再测试试一下nobody:

cd ../../5_Simulations/nbody/
make

执行:

./nbody -benchmark -numbodies=256000 -device=0

得到:

> Windowed mode
> Simulation data stored in video memory
> Single precision floating point simulation
> 1 Devices used for simulation
gpuDeviceInit() CUDA Device [0]: “GeForce GTX 1080
> Compute 6.1 CUDA device: [GeForce GTX 1080]
number of bodies = 256000
256000 bodies, total time for 10 iterations: 2291.469 ms
= 286.000 billion interactions per second
= 5719.998 single-precision GFLOP/s at 20 flops per interaction

参考:
Nvidia GTX 1080 on Ubuntu 16.04 for Deep Learning
Ubuntu 16.04下安装Tensorflow(GPU)
ubuntu16.04安装 cuda7.5
Ubuntu16.04无法安装CUDA吗?
Ubuntu16.04+matlab2014a+anaconda2+OpenCV3.1+caffe安装
ubuntu 16.04 编译opencv3.1,opencv多版本切换
TensorFlow, Caffe, Chainer と Deep Learning大御所を一気に source code build で GPU向けに setupしてみた
feature request: support for cuda 8.0 rc
GTX 1080 CUDA performance on Linux (Ubuntu 16.04) preliminary results (nbody and NAMD)
Anyone able to run Tensorflow with 1070/1080 on Ubuntu 16.04/15.10/15.04?
Tensorflow on Ubuntu 16.04 with Nvidia GTX 1080

注:原创文章,转载请注明出处及保留链接“我爱自然语言处理”:http://www.52nlp.cn

本文链接地址:深度学习主机环境配置: Ubuntu16.04+Nvidia GTX 1080+CUDA8.0 http://www.52nlp.cn/?p=9226

此条目发表在深度学习分类目录,贴了, , , , , , , , , , , , , , , , , , , , 标签。将固定链接加入收藏夹。

深度学习主机环境配置: Ubuntu16.04+Nvidia GTX 1080+CUDA8.0》有 26 条评论

  1. iresine说:

    请问,虚拟机+16.04装完nvidia驱动后循环登录界面怎么解决?

    [回复]

    52nlp 回复:

    驱动没有安装正确?能否切换到命令行模式登录重新安装驱动

    [回复]

    52nlp 回复:

    参考一下这个帖子 https://www.v2ex.com/t/275534 ,是否安装的是16.04自带的nvidia-361驱动,安装这个貌似有点问题

    [回复]

    FF 回复:

    虚拟机是不能装cuda的。虚拟机的显卡是虚拟出来的,不是nvidia的显卡

    [回复]

  2. soulmachine说:

    我也碰到了循环登录的问题,装完nvidia驱动后开机输入密码,会闪一下又返回登录界面,我是真机装的

    [回复]

    soulmachine 回复:

    解决了,必须要禁用主板的 Secure Boot

    [回复]

  3. soulmachine说:

    Driver: Not Selected
    Toolkit: Installation Failed. Using unsupported Compiler.
    Samples: Cannot find Toolkit in /usr/local/cuda-8.0

    [回复]

    52nlp 回复:

    这一步安装cuda失败了,不过这个提示看不出哪一步出了问题。

    [回复]

  4. S. Qu说:

    Thank you. Before I saw your post I spent a lot of time trying to make the driver workingissue with the driver; you PPA approach solved my problem.

    On the following issue:
    “这里如果提示gcc版本过高,可以安装低版本的gcc并做软连接替换,具体方法请自行google,我用低版本的gcc4.9替换了ubuntu16.04自带的gcc5.x版本”
    I downloaded and ran nvidia’s fix file (sudo sh cuda_8.0.27.1_linux.run) after installation of cuda 8, and it worked. This fix might not be available when you wrote your post.

    Thanks again.

    [回复]

    52nlp 回复:

    谢谢提示

    [回复]

  5. bbXU说:

    export 只能临时使用,重启后又无效了,,,

    [回复]

    52nlp 回复:

    当然要写进 ~/.bashrc 里啊

    [回复]

  6. keith说:

    Hello.
    i am new to tensorflow, and spent a lot of days to install tensorflow, but it didn’t work. Need help.

    sudo ./configure
    ~/tensorflow ~/tensorflow
    Please specify the location of python. [Default is /usr/bin/python]:
    Do you wish to build TensorFlow with Google Cloud Platform support? [y/N] n
    No Google Cloud Platform support will be enabled for TensorFlow
    Found possible Python library paths:
    /usr/local/lib/python2.7/dist-packages
    /usr/lib/python2.7/dist-packages
    Please input the desired Python library path to use. Default is [/usr/local/lib/python2.7/dist-packages]

    /usr/local/lib/python2.7/dist-packages
    Do you wish to build TensorFlow with GPU support? [y/N] y
    GPU support will be enabled for TensorFlow
    Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]:
    Please specify the Cuda SDK version you want to use, e.g. 7.0. [Leave empty to use system default]: 8.0
    Please specify the location where CUDA 8.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
    Please specify the Cudnn version you want to use. [Leave empty to use system default]: 4
    Please specify the location where cuDNN 4 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
    Please specify a list of comma-separated Cuda compute capabilities you want to build with.
    You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
    Please note that each additional compute capability significantly increases your build time and binary size.

    Found stale PID file (pid=3471). Server probably died abruptly, continuing…
    ..
    INFO: Starting clean (this may take a while). Consider using –expunge_async if the clean takes more than several minutes.
    .
    ERROR: /home/keithyin/tensorflow/tensorflow/contrib/session_bundle/BUILD:134:1: no such target ‘//tensorflow/core:android_lib_lite’: target ‘android_lib_lite’ not declared in package ‘tensorflow/core’ defined by /home/keithyin/tensorflow/tensorflow/core/BUILD and referenced by ‘//tensorflow/contrib/session_bundle:session_bundle_lite’.
    ERROR: /home/keithyin/tensorflow/tensorflow/core/platform/default/build_config/BUILD:56:1: no such package ‘@jpeg_archive//’: Error downloading from http://www.ijg.org/files/jpegsrc.v9a.tar.gz to /home/keithyin/.cache/bazel/_bazel_root/9192340d7b606ddb9ea35b29a97154c1/external/jpeg_archive: Error downloading http://www.ijg.org/files/jpegsrc.v9a.tar.gz to /home/keithyin/.cache/bazel/_bazel_root/9192340d7b606ddb9ea35b29a97154c1/external/jpeg_archive/jpegsrc.v9a.tar.gz: Connection timed out and referenced by ‘//tensorflow/core/platform/default/build_config:platformlib’.
    ERROR: Evaluation of query “deps((//… union @bazel_tools//tools/jdk:toolchain))” failed: errors were encountered while computing transitive closure.
    Configuration finished

    [回复]

    52nlp 回复:

    you can refer this issue: https://github.com/tensorflow/tensorflow/issues/4312

    [回复]

  7. 洪伟说:

    我的配置是华硕z170a,CPU是i7 6700k,显卡是GTX1080,想安装Ubuntu系统,安装完重启后就一个桌面一个鼠标,在右键打开终端后在source文件中添加源,然后安装英伟达的驱动,再重启后就一直在登录界面死循环,闪一下又回到登录界面,咋回事啊,有没有配置成功的大神指点一下迷津,我邮箱是425591614@qq.com,求救

    [回复]

    马里奥赛德 回复:

    在安装驱动之前试试更新下软件和系统,保证系统最新,有时候会出现兼容问题的

    [回复]

  8. kun说:

    你好,1080/ubuntu 14.04,驱动都装利索了。跑mnist的例程,prototxt定义GPU,最后 loss 2.3, acc只有0.113。用CPU跑的话acc 0.99,求解这是怎么回事儿…网上找不到相关信息

    [回复]

    52nlp 回复:

    从你这个描述来看我也没有办法帮你解决,至少给一些报错信息或者warning之类的log信息。

    [回复]

  9. 马继昌说:

    你好,安装好Ubuntu16.04后,重启进入grub,按住e进行编辑,将quiet splash $vt_handoff 改为quiet splash nomodeset 但是还是会出现紫色屏幕,然后黑屏,紧接着会提示“the input tming is not supported……..” ,这个问题怎样去解决呢?

    [回复]

  10. Away说:

    我想使用windows10的仿Linux子系统配置,不知道可行吗。周末试下,真行就好了,还能用vs。。

    [回复]

  11. Away说:

    我想使用windows10自带的仿Linux子系统配置,不知道可行吗。周末试下,真行就好了,还能用vs。。

    [回复]

    52nlp 回复:

    抱歉,不清楚这个仿Linux子系统配置

    [回复]

  12. 杨振说:

    博主,您好,请问您的根目录、home目录都是是挂载在固态硬盘嘛。

    [回复]

    52nlp 回复:

    是的

    [回复]

    杨振 回复:

    感谢回复,还想问下,你机械硬盘挂载在哪个目录?
    然后网上有说,固态硬盘用来放训练数据会好点,是不是将固态硬盘挂载成数据盘的意思?

    [回复]

    52nlp 回复:

    /opt目录;目前一般在SSD上训练模型,一些数据放在/opt目录下,数据量还不算大,所以基本上ok。

发表评论

电子邮件地址不会被公开。 必填项已用*标注