pip Archives - Jack Stromberg

Per Microsoft: Some packages may not install using pip when run on Azure. It may simply be that the package is not available on the Python Package Index. It could be that a compiler is required (a compiler is not available on the machine running the web app in Azure App Service).

Example, you may receive an error like this when trying to install a specific package (in this case, trying to install Pandas):

Command: "D:\home\site\deployments\tools\deploy.cmd"
Handling python deployment.
KuduSync.NET from: 'D:\home\site\repository' to: 'D:\home\site\wwwroot'
Copying file: 'requirements.txt'
Detected requirements.txt.  You can skip Python specific steps with a .skipPythonDeployment file.
Detecting Python runtime from runtime.txt
Detected python-2.7

Found compatible virtual environment.
Pip install requirements.
Downloading/unpacking Flask==0.12.1 (from -r requirements.txt (line 1))
Downloading/unpacking numpy==1.15.0rc2 (from -r requirements.txt (line 2))
Downloading/unpacking pandas==0.22.0 (from -r requirements.txt (line 3))
  Running setup.py (path:D:\home\site\wwwroot\env\build\pandas\setup.py) egg_info for package pandas

    Could not locate executable g77
    Could not locate executable f77
    Could not locate executable ifort
    Could not locate executable ifl
    Could not locate executable f90
    Could not locate executable efl
    Could not locate executable gfortran
    Could not locate executable f95
    Could not locate executable g95
    Could not locate executable effort
    Could not locate executable efc
    don't know how to compile Fortran code on platform 'nt'
    non-existing path in 'numpy\\distutils': 'site.cfg'
    Running from numpy source directory.

    d:\local\temp\easy_install-dsrz9g\numpy-1.15.0rc2\setup.py:385: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
      run_build = parse_setuppy_commands()
    D:\python27\Lib\distutils\dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
      warnings.warn(msg)
    d:\local\temp\easy_install-dsrz9g\numpy-1.15.0rc2\numpy\distutils\system_info.py:625: UserWarning:
        Atlas (http://math-atlas.sourceforge.net/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [atlas]) or by setting
        the ATLAS environment variable.
      self.calc_info()

    d:\local\temp\easy_install-dsrz9g\numpy-1.15.0rc2\numpy\distutils\system_info.py:625: UserWarning:
        Blas (http://www.netlib.org/blas/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [blas]) or by setting
        the BLAS environment variable.
      self.calc_info()

    d:\local\temp\easy_install-dsrz9g\numpy-1.15.0rc2\numpy\distutils\system_info.py:625: UserWarning:
        Blas (http://www.netlib.org/blas/) sources not found.
        Directories to search for the sources can be specified in the
        numpy/distutils/site.cfg file (section [blas_src]) or by setting
        the BLAS_SRC environment variable.
      self.calc_info()

    d:\local\temp\easy_install-dsrz9g\numpy-1.15.0rc2\numpy\distutils\system_info.py:625: UserWarning:
        Lapack (http://www.netlib.org/lapack/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [lapack]) or by setting
        the LAPACK environment variable.
      self.calc_info()

    d:\local\temp\easy_install-dsrz9g\numpy-1.15.0rc2\numpy\distutils\system_info.py:625: UserWarning:
        Lapack (http://www.netlib.org/lapack/) sources not found.
        Directories to search for the sources can be specified in the
        numpy/distutils/site.cfg file (section [lapack_src]) or by setting
        the LAPACK_SRC environment variable.
      self.calc_info()

    D:\python27\Lib\distutils\dist.py:267: UserWarning: Unknown distribution option: 'define_macros'
      warnings.warn(msg)
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "D:\home\site\wwwroot\env\build\pandas\setup.py", line 743, in <module>
        **setuptools_kwargs)
      File "D:\python27\Lib\distutils\core.py", line 111, in setup
        _setup_distribution = dist = klass(attrs)
      File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\dist.py", line 262, in __init__
        self.fetch_build_eggs(attrs['setup_requires'])
      File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\dist.py", line 287, in fetch_build_eggs
        replace_conflicting=True,
      File "D:\home\site\wwwroot\env\lib\site-packages\pkg_resources.py", line 614, in resolve
        dist = best[req.key] = env.best_match(req, ws, installer)
      File "D:\home\site\wwwroot\env\lib\site-packages\pkg_resources.py", line 857, in best_match
        return self.obtain(req, installer)
      File "D:\home\site\wwwroot\env\lib\site-packages\pkg_resources.py", line 869, in obtain
        return installer(requirement)
      File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\dist.py", line 338, in fetch_build_egg
        return cmd.easy_install(req)
      File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\command\easy_install.py", line 613, in easy_install
        return self.install_item(spec, dist.location, tmpdir, deps)
      File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\command\easy_install.py", line 643, in install_item
        dists = self.install_eggs(spec, download, tmpdir)
      File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\command\easy_install.py", line 833, in install_eggs
        return self.build_and_install(setup_script, setup_base)
      File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\command\easy_install.py", line 1055, in build_and_install
        self.run_setup(setup_script, setup_base, args)
      File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\command\easy_install.py", line 1043, in run_setup
        raise DistutilsError("Setup script exited with %s" % (v.args[0],))
    distutils.errors.DistutilsError: Setup script exited with error: Microsoft Visual C++ 9.0 is required (Unable to find vcvarsall.bat). Get it from http://aka.ms/vcpython27
    Complete output from command python setup.py egg_info:

Could not locate executable g77
Could not locate executable f77
Could not locate executable ifort
Could not locate executable ifl
Could not locate executable f90
Could not locate executable efl
Could not locate executable gfortran
Could not locate executable f95
Could not locate executable g95
Could not locate executable effort
Could not locate executable efc

don't know how to compile Fortran code on platform 'nt'
non-existing path in 'numpy\\distutils': 'site.cfg'
Running from numpy source directory.
d:\local\temp\easy_install-dsrz9g\numpy-1.15.0rc2\setup.py:385: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
  run_build = parse_setuppy_commands()
D:\python27\Lib\distutils\dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
  warnings.warn(msg)

d:\local\temp\easy_install-dsrz9g\numpy-1.15.0rc2\numpy\distutils\system_info.py:625: UserWarning:
    Atlas (http://math-atlas.sourceforge.net/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [atlas]) or by setting
    the ATLAS environment variable.
  self.calc_info()

d:\local\temp\easy_install-dsrz9g\numpy-1.15.0rc2\numpy\distutils\system_info.py:625: UserWarning:
    Blas (http://www.netlib.org/blas/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [blas]) or by setting
    the BLAS environment variable.
  self.calc_info()

d:\local\temp\easy_install-dsrz9g\numpy-1.15.0rc2\numpy\distutils\system_info.py:625: UserWarning:
    Blas (http://www.netlib.org/blas/) sources not found.
    Directories to search for the sources can be specified in the
    numpy/distutils/site.cfg file (section [blas_src]) or by setting
    the BLAS_SRC environment variable.
  self.calc_info()

d:\local\temp\easy_install-dsrz9g\numpy-1.15.0rc2\numpy\distutils\system_info.py:625: UserWarning:
    Lapack (http://www.netlib.org/lapack/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [lapack]) or by setting
    the LAPACK environment variable.
  self.calc_info()

d:\local\temp\easy_install-dsrz9g\numpy-1.15.0rc2\numpy\distutils\system_info.py:625: UserWarning:
    Lapack (http://www.netlib.org/lapack/) sources not found.
    Directories to search for the sources can be specified in the
    numpy/distutils/site.cfg file (section [lapack_src]) or by setting
    the LAPACK_SRC environment variable.
  self.calc_info()

D:\python27\Lib\distutils\dist.py:267: UserWarning: Unknown distribution option: 'define_macros'
  warnings.warn(msg)

Traceback (most recent call last):
  File "<string>", line 17, in <module>
  File "D:\home\site\wwwroot\env\build\pandas\setup.py", line 743, in <module>
    **setuptools_kwargs)
  File "D:\python27\Lib\distutils\core.py", line 111, in setup
    _setup_distribution = dist = klass(attrs)
  File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\dist.py", line 262, in __init__
    self.fetch_build_eggs(attrs['setup_requires'])
  File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\dist.py", line 287, in fetch_build_eggs
    replace_conflicting=True,
  File "D:\home\site\wwwroot\env\lib\site-packages\pkg_resources.py", line 614, in resolve
    dist = best[req.key] = env.best_match(req, ws, installer)
  File "D:\home\site\wwwroot\env\lib\site-packages\pkg_resources.py", line 857, in best_match
    return self.obtain(req, installer)
  File "D:\home\site\wwwroot\env\lib\site-packages\pkg_resources.py", line 869, in obtain
    return installer(requirement)
  File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\dist.py", line 338, in fetch_build_egg
    return cmd.easy_install(req)
  File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\command\easy_install.py", line 613, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\command\easy_install.py", line 643, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\command\easy_install.py", line 833, in install_eggs
   return self.build_and_install(setup_script, setup_base)
  File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\command\easy_install.py", line 1055, in build_and_install
    self.run_setup(setup_script, setup_base, args)
  File "D:\home\site\wwwroot\env\lib\site-packages\setuptools\command\easy_install.py", line 1043, in run_setup
    raise DistutilsError("Setup script exited with %s" % (v.args[0],))

distutils.errors.DistutilsError: Setup script exited with error: Microsoft Visual C++ 9.0 is required (Unable to find vcvarsall.bat). Get it from http://aka.ms/vcpython27

----------------------------------------

Cleaning up...

Command python setup.py egg_info failed with error code 1 in D:\home\site\wwwroot\env\build\pandas
Storing debug log for failure in D:\home\pip\pip.log

An error has occurred during web site deployment.
\r\nD:\Program Files (x86)\SiteExtensions\Kudu\75.10629.3460\bin\Scripts\starter.cmd "D:\home\site\deployments\tools\deploy.cmd"

This guide is a reflection on how to use Wheel files to install Modules that cannot natively be installed via pip due to a compiler missing in the Azure App Service:

Microsoft Official documentation can be found here: https://docs.microsoft.com/en-us/azure/app-service/web-sites-python-configure#troubleshooting---package-installation

Tutorial

Modify requirements.txt file
1. Add the following item as the first line to the document:
  1. --find-links wheelhouse
    1. Note: If you do not have a requirements.txt file, you can simply create a new text document and add this line to it. The requirements.txt file is what allows the Azure App Service to automatically go out and try and download packages you may need for your application. Official documentation on this file is found here: https://docs.microsoft.com/en-us/azure/app-service/web-sites-python-configure#package-management
2. Navigate to the Kudu Debug Console by going to https://yourappservice.scm.azurewebsites.net/DebugConsole
3. Within the debug console, navigate to your version of Python.
  1. Note: The default Python versions in an Azure App Service are 2.7 and 3.4; however since Wheel will need to install some files, you cannot leverage the default directories of D:\Python27 for v2.7 and D:\Python34 for v3.4
  2. In this case, I'd recommend leveraging Extensions to install whatever version of Python. Documentation on this can be found here: https://blogs.msdn.microsoft.com/pythonengineering/2016/08/04/upgrading-python-on-azure-app-service/
4. Install the Python Wheel module:
  1. python.exe -m pip install wheel
5. Obtain Wheel files
  1. Option 1: Build your own wheel files
    1. Execute the following command:
      1. python.exe -m pip wheel -r D:\home\site\wwwroot\requirements.txt -w wheelhouse
  2. Option 2: Obtain Wheel files
    1. Create a wheelhouse folder within your python directory
      1. mkdir wheelhouse
    2. Copy whl files to this directory
      1. You can obtain wheel files from PyPi or from Laboratory for Fluorescence Dynamics, University of California, Irvine.
        
        PyPi: Search for the module and then clicking on the Download Files button
        
        https://pypi.org/
        
        Laboratory for Fluorescence Dynamics, University of California, Irvine: Simply download the appropriate whl file listed on the page below
        
        https://www.lfd.uci.edu/~gohlke/pythonlibs/
6. Install Modules
  1. Manual Install
    1. Execute the following command:
      python.exe -m pip install --upgrade -r D:\home\site\wwwroot\requirements.txt
  2. Deployment Install (from CI/CD pipeline)
    1. Configure .deployment and deploy.cmd file
      1. Official documentation on this can be found here: https://github.com/projectkudu/kudu/wiki/Custom-Deployment-Script
      2. .deployment file
        
        [config]
        command = deploy.cmd
      3. deploy.cmd file (modify the python directory to reflect your version)
        
        :: 1. Install Wheel
        echo Configure Wheel
        D:\home\python364x64\python.exe -m pip install wheel:: 2. Install packages
        echo Pip install requirements.
        D:\home\python364x64\python.exe -m pip install --upgrade -r D:\home\site\wwwroot\requirements.txt

At this point, the modules in question should be installed and ready for use! 🙂

Jack Stromberg

A site about stuff

Tag Archives: pip

Installing Python Wheel files on an Azure App Service

Tutorial