I've been supporting Drupal VM (a local Drupal CMS development environment) for Windows, Mac, and Linux for the past couple years, and have been using Vagrant and virtual machines for almost all my development (mostly PHP, but also some Python and Node.js at the moment) for the past four years. One theme that comes up quite frequently when dealing with VMs, open source software stacks (especially Drupal/LAMP), and development, is how much extra effort there is to make things work well on Windows.
Problem: tool-builders use Linux or macOS
The big problem, I see, is that almost all the tool-builders for OSS web software run either macOS or a flavor of Linux, and many don't even have access to a Windows PC (outside of maybe an odd VM for testing sites in Internet Explorer or Edge, if they're a designer/front-end developer). My evidence is anecdotal, but go to any OSS conference/meetup and you'll likely see the same.
When tool-builders don't use Windows natively, and in many cases don't even have access to a real Windows environment, you can't expect the tools they build to always play nice in that kind of environment. Which is why virtualization is almost an essential component of anyone's Windows development workflow.
However, that's all a bit of an aside leading up to the substance of this post: common issues with Windows-based development using virtual machines (e.g. VirtualBox, Hyper-V, VMware, etc.), and some solutions or tips for minimizing the pain.
As an aside, I bought a Lenovo T420 and stuck 2 SSDs and an eMMC card in it, then I purchased and installed copies of Windows 7, 8, and 10 on them so I could make sure the tools I build work at least decently well on Windows in multiple different environments. Many open source project maintainers aren't willing to fork over a $500+ outlay just to test in a Windows environment, especially since the Windows bugs often take 2-4x more time and cause many more grey hairs than similar bugs on Linux/macOS.
Tips for more efficiency on Windows
First: if there's any way you can use Linux or Mac instead of Windows, you'll be in much less pain. This is not a philosophical statement, nor is it an anti-Microsoft screed. Almost all the modern web development toolkits are supported primarily (and sometimes only) for Linux/POSIX-like environments. Getting everything in a modern stack working together in Windows natively is almost never easy; getting things working within a VM is a viable but sometimes annoying alternative.
Second: if you can do all development within the VM, you'll be in somewhat less pain. If you can check out your codebase inside the VM's filesystem (and not use a synced folder), then edit files in the VM, or edit files in Windows through a mounted share (instead of using built-in shared folders), some of the pain will be mitigated.
Here are some of the things I've found that make development on Windows possible; and sometimes even enjoyable:
Using a POSIX-like environment
2019 Update: The Windows Subsystem for Linux is now the recommended way to have a Linux environment running on your Windows 10 or later machine. Note, however, that it cannot be used in all scenarios and still carries with it some caveats, especially having to do with controlling Windows executables and files from the Linux subsystem (and vice-versa).
I'd recommend using either Cygwin (with openssh) or Cmder instead of PowerShell/Git Bash/Git Shell. These solutions emulate the same command line environment you get on Linux/BSD much more closely than PowerShell, Git Bash, or Git Shell, and things like SSH and other commands you'll see in guides/tutorials work much more consistently in these environments.
Also, for many problems (especially permissions or filesystem-related), running your terminal environment as administrator seems to help.
Working with symbolic links (symlinks)
If you have to support symbolic links inside the VM, you can't use Git Bash, Git Shell, or Powershell when managing the Vagrant environment (there is a way, but it's excruciating). It's highly recommended to use either Cygwin (with openssh) or Cmder instead (see above). There are additional caveats when you require symlink support:
- You need to run Cygwin or Cmder as administrator (otherwise you can't create symlinks).
- You have to do everything inside the VM (you can do git stuff outside, and edit code outside, but anything you need to do dealing with creating/changing/deleting symlinks should be done inside the VM).
- If you touch the symlinks outside the VM, bad things will happen.
- Symlinks only work with SMB or native shares (possibly rsync, too, but that's a bit harder to work with in my experience.
- If you switch from native to SMB, or vice-versa, you have to rebuild all symlinks (symlinks between the two synced folder types are incompatible).
- If you use SMB, you have to set custom mount options for Vagrant in the synced folder configuration, e.g.:
mount_options: ["mfsymlinks,dir_mode=0755,file_mode=0755"]
VirtualBox Guest Additions
Probably half the problems I've seen are due to outdated (or not-installed) VirtualBox Guest Additions. Many Vagrant box maintainers don't update their boxes regularly, and if this is the case, and you have a newer version of VirtualBox, all kinds of strange issues (ssh errors, synced folder errors, etc.) ensue.
I highly recommend (for Mac, Linux, and Windows) you install the vagrant-vbguest plugin: vagrant plugin install vagrant-vbguest
Delete a deep folder hierarchy (nested directories)
For many of the projects I've worked on, folder hierarchies can get quite deep, e.g. C:/Users/jgeerling/Documents/GitHub/myproject/docroot/sites/all/themes/mytheme/node_modules/dist/modulename/etc. Windows hates deep folder hierarchy, and if you try deleting a folder with such a structure either in Explorer or using rmdir in PowerShell, you'll likely end up with an error message (e.g. File name too long...).
To delete a folder with a deep hierarchy (many nested directories), you need to install robocopy (part of the Windows Server set of tools), then follow these directions to delete the directory.
Node.js and npm problems
There are myriad issues running Node.js, NPM, and the ecosystem of associated build tools. It's hard enough keeping things straight with multiple Node versions and nvm on Mac/Linux... but toss in a Windows environment and most corporate networks/group policies, and you will also need to deal with:
- If you have certain flavors of antivirus running, you might have trouble with Node.js and NPM.
- If you are behind a corporate proxy, you will need to run a few extra commands to make sure things like
aptwork inside the VM.
If you attempt to use Node/NPM within Windows, you should run Cygwin or Cmder as administrator, and possibly disable AntiVirus software. If working behind a proxy, you will also need to configure NPM to work with the proxy (in addition to configuring the VM/Linux in general to work behind the proxy):
$ npm config set proxy http://username:password@host:port/
$ npm config set https-proxy http://username:password@host:port/Intel VT-x virtualization
Many PC laptops (especially those from Lenovo, HP, and Dell) have Intel's VT-x virtualization turned off by default, which can cause issues with many Vagrant boxes. Check your computer manufacturer's knowledge base for instructions for enabling VT-x in your system BIOS/UEFI settings.
I have a Lenovo T420, and had to follow these instructions for enabling virtualization from Lenovo's support site.
Other Notes
I've also compiled a list of Windows tips and tricks in the Drupal VM project documentation: Drupal VM Docs - Windows Notes.
Summary
Developing for Drupal with Vagrant and VMs on Windows is possible—I've used Drupal VM and related projects with Windows 7, 8, and 10, with and without proxies, on a variety of hardware—but not optimal for all cases. If you keep running into issues like those listed above (or others), you might want to investigate switching your development environment to Linux or macOS instead.
Comments
Hi Jeff,
You helped me with windows issue a while back (thanks a lot!). When I saw the title of this post my heart stopped as I thought you were about to announce you were stopping supporting Windows altogether.
So here is the thing: My PC at home is on Windows. Sure I could switch to Linux (or dual boot or simply have a dedicated Linux dev machine), but I'm comfortable with Windows due to countless software (graphics stuff, games, etc) that are less prevalent on Linux or Mac... So it's a choice of convenience.
So by using Windows, I understand that to any Rockstar Webdev™ I am basically an uncultured heathen with leprosy from the traveling carnival and do not warrant a millisecond of attention. Which is why I'm very grateful of your attention/troubleshooting and support of Windows on Drupal VM!
But here is my personal 2 cents regarding my personal use case of Drupal VM:
The reason I use drupalvm is because I use windows and need an easy linux dev environment for Drupal. I'm not sure if I would be using a vm at all if I ran Linux as my main dev machine. Sure it can abstract things and help you experiment without breaking your main machine. But otherwise it seems like adding another layer of complexity.
So what Drupal VM is giving me is a good compromise of not having to spend my days fiddling with bad Linux drivers or incompatible software and keeping my comfortable and well known environment while still using Linux tools for development.
To achieve that I had to make some compromises:
Maybe I'm weird.. But it kinda works. Of course I live in constant fear that the whole vm environment will one day simply choke and die...
But Thank Jeff for your hard work and dedication!
I don't think you're weird at all :)
What you're doing is likely the most efficient/least painful way to do it all within a Windows environment—you bypass the myriad issues of synced folders by doing it all within the VM's filesystem. Thanks for commenting with the details!
+1
That is exactly the same experience and solution I have.
Sir I applaud TO YOU!
Muchas gracias senor !:)
Everyone seems to talk about how much faster SMB is with Vagrant if you're using Windows 10, specifically for PHP projects using Composer (I think Node projects may be worse than Composer's vendor directory but the same idea is there). Setting PHPStorm to automatically upload and sync your files between the VM and the host really seems like the best option for Windows devs.
Thanks for supporting the Windows users. All too often we do feel looked down upon. I started developing for the web on Windows way back when that is where all the tools were. It is important to remember that even with the rise of mobile, Windows users are still the dominant base for many websites.
Admittedly, I haven't yet tried the DrupalVM, but mean to. I promise! I primarily use Windows as a GUI, but all dev work runs in Linux VMs. I am a big fan of using VMs, regardless of the desktop environment. I had been a mixed user up till about 10 yrs ago, and finally gave up on Apple. As the previous commenter mentions, Windows is key for me to use a multitude of various software packages that are not available on Linux or Mac, and Apple has a serious deficiency in the powerhouse desktop environment, and support for powerful video cards.
VMs are a critical tool for development. Linux is the dominant force in hosting, and many of those hosts are using VMs as well. So developing in an environment as close as possible to the live host helps reduce "works on my machine" syndrome. It also eliminates the possibility of an errant command or update from bringing down the whole system. Introducing regular snapshots of the VM can then separate backups of the dev environment from the desktop environment. So restoring one, doesn't impact the other. Everyone is backing up, right? It also allows for less stressful experimentation. Even when I use a Linux desktop, I still use Ubuntu Server VMs for development.
Hyper-V is my preferred tool to manage my VMs on a dedicated server. The bonus of using a dedicated hypervisor is multiple client machines can easily be used for accessing the same dev environment, and snapshots are easily managed and even backed up offsite nightly. Since a dedicated hypervisor isn't getting used for anything else, it remains incredibly stable for long periods of time. The down side of course is it is another piece of equipment to purchase and take up space. I also run Hyper-V on my Win8 laptop to have a Linux environment available on the go. I do also us Virtualbox occasionally on my Win7 desktop to fire up environments that will not run consistently, such as various ISOs Microsoft offers for Windows/IE versions.
Accessing files between environments has been fairly simple for me. I use Samba in the VMs and map the drives in Windows. This allows easy use of whatever tools I want on Windows to directly work with the Linux file system. There are probably better tools, but good old PuTTy has worked just fine to bring the Linux command line into Windows. PuTTy is also the too I use if I need to work on the command line of a remote server, so it keeps things familiar.
The most frustrating thing I tend to encounter with this setup has nothing to do with Windows; rather the myopic view of Macintosh based developers that assume everyone is developing directly on a Mac. All too often I encounter tutorials that only document something that has parity on Linux in a way that only works on a Mac, using tools like Homebrew for example. I understand the draw of such tools, we all want fast and easy. But this excludes Linux, the environment where their code will most likely eventually live.
Exactly; if anything, base tutorials on the main Linux distro you're targeting, not at a developer environment. Bonus points for targeting the latest release of both a Debian and RHEL derivative, to cover like 95% of the market.
Great article Jeff. As a long time mac dev I've recently had to go through the pain of trying to get a development environment working on Windows at my current job. From my experience it can be done it's just harder. The Drupal VM project has definitely been a huge win for the team I work on. Thanks.
Long Path Fixer presents you with a simple list of files and folders in the current directory (including “hidden” files and folders). You can drag and drop files or folders onto it and it will navigate directly to path of whatever you dropped.
Hi Jeff, you helped me out with Drupal VM in 2015 at the Chicago MidCamp. I've been using DrupalVM for all of my Drupal projects since (I use box.scotch.io for non-Drupal projects). I wanted to say thank you for genuinely supporting Windows users. I have been a Windows user for 20 years. Windows happened to be what I started with, what I can afford, and what I've stayed with.
The Drupal community is so diverse and welcomes everyone; all people, ethnicity, ability, and orientation... yet it's still socially acceptable to poke fun at Windows users :(. Sorry, but not everyone can afford $1,000+ development grade Mac. I got so tired of getting hassled for using Windows that, now when I go to any public event, I use my dual-boot Linux laptop instead just so I don't have to hear any crap about "oh, you're on Windows, of course it doesn't work, sorry I can't help you.". Windows does work, it works very well for development with virtual machines, and yes, often with these caveats you have described.
DrupalVM works so well on Windows, that now I hope people will instead say "oh, I see you're on Windows? Have you tried DrupalVM? It will solve most of the issues you're having. Let me help get you started." Hopefully this will reduce the instances of discouraging brand new potential contributors who had the courage to show up to their first Drupal event ready to help, only to be dismissed for using Windows like they may as well have had leprosy.
Thank you again for supporting Windows, for this reason (plus outstanding documentation and support) is what makes you a true professional.
--TG
I too appreciate Jeff's work on Windows but I have to say that I had DrupalVM up and running on OS X very quickly — and this is absolutely not the case with Windows. This is not to point fingers but to underline his point that developing on OS X or Linux may save people days of work and much frustration.
It feels like I have hit every possible problem there is. Perhaps this list will help someone to get things to work on their Windows machine (10, specifically).
Here they are, in no particular order:
- VT-x was off; turn that on
- Hyper V in Windows was on; turn that off
- turn off Nested Paging
- deep folders on Windows (several different workarounds, Jeff recommends robocopy, which I haven't tried)
- install vagrant-proxy to get through the corporate firewall
- disable "Enable I/O APIC" on VirtualBox (but it won't stay off)
- install Visual C++ SP1 to complete the install of Vagrant on Windows 10
- add "KexAlgorithms diffie-hellman-group1-sha1" to get ssh working from Cygwin (but not totally functioning yet)
- turning off hardware video acceleration on VirtualBox.
Now I'm spending time on the error "The guest machine entered an invalid state while waiting for it to boot." I can boot the vm manually through VirtualBox when I turn off Enable I/O APIC but the switch won't stay off.
It's a shame because Jeff actually *has* put valuable work on getting DrupalVM on Windows working.
But I certainly can't recommend the setup at this time. And, unfortunately, I can't order 10 new Macs for the team.
LOL. Just stumbled upon your post 3 months into my project. I have hit on every single issues you mentioned. Nodejs components requiring symlinks is my latest roadblock. Editing source from vagrant mount point no longer works in that scenario. I am not sure if it is even worth trying adding X and run IntelliJ from VMs. I used to be an emacs guru back 1992. But I can't pull that off anymore, so spoiled by IDEs.
Hi
I realise this is a relatively old article, but it is still very useful for myself as I am considering migrating from puphpet to DrupalVM.
I agree with your suggestion that the best bet is to work soley in the virtual machine when Windows is the host machine. In puphpet I disable the shared folder completely, but set up samba so I can browse into the virtual machine and edit files as if they were in a local folder. I cannot work out how to do this in DrupalVM. Can you explain in a bit more detail?
So far I have copied the default.config.yaml to config.yaml and modified 'vagrant_synced_folders', but without luck:
- local_path: .
destination: /var/www/drupalvm
type: ""
create: false
options_override:
disabled: true
Thanks
On Windows 10 if you want to mount shared folder as smb type specify ip-adress of host machine, and add extra mount options is vers=3.0=>
sync_type: smb
smb:
smb_host: '192.168.56.1'
smb_username: alex
smb_password: password
mount_options:
dir_mode: '0775'
file_mode: '0775'
vers: '3.0'