This post was updated 2486 days ago and some of the ideas may be out of date.

前几天玩了下python爬虫Scrapy,用md记录了一篇在OS X上的安装过程,现在抽空分享给大家!(悲剧的是当时写文章的截图当做垃圾删掉了……)

一、打开终端执行命令

sudo easy_install pip

二、终端执行命令安装Scrapy

sudo pip install Scrapy

三、解决错误

安装成功当然是最好的,但是往往是安装失败,或许你会看到如下提示

Command "/usr/bin/python -c "import setuptools,     tokenize;__file__='/private/tmp/pip-build-9RYtLC/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-544HZx-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/tmp/pip-build-9RYtLC/lxml

解决方法:

测试标题

  • 1.使用lxml命令安装
    STATIC_DEPS=true pip install lxml
  • 2.如果还是失败,请安装命令行开发工具(安装有Xcode的同学可以省略这步)
    xcode-select --install
    然后执行安装命令。

  • 3.如果还是失败,请配置libxml路径(注意系统版本号)

sudo ln -s  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/libxml2/libxml/ /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/libxml

然后执行安装命令。

四、安装成功

安装成功后,在终端输入:scrapy

显示如下

bogon:~ ian$ scrapy
Scrapy 1.0.3 - no active project

Usage:
  scrapy  [options] [args]

Available commands:
  bench         Run quick benchmark test
  commands      
  fetch         Fetch a URL using the Scrapy downloader
  runspider     Run a self-contained spider (without creating a project)
  settings      Get settings values
  shell         Interactive scraping console
  startproject  Create new project
  version       Print Scrapy version
  view          Open URL in browser, as seen by Scrapy

  [ more ]      More commands available when run from project directory

Use "scrapy  -h" to see more info about a command

大功告成!另外请不要删除Mac自带的python,否则你会后悔的,详情请看我的这篇博文 ->> 传送门