Magento

Crazy Magento 2 core patches

Magento

Last year I took over a maintenance of the Magento 2 Commerce site. One of first tasks was adjusting the site to meet industry standards as it was a bit messy. The whole content of vendor/ directory was kept in the repository and there were about 30 patch files placed in the site root, without any information which were applied and in which order. Most of them looked as provided by M2 support. I needed to clean this up – to figure out which patches are applied, in which order they are applied and finally find a way allowing me to remove vendor/ from the repository and to be able to apply patches dynamically, during the deployment.

This was a challenging task – I needed to keep comparing the vendor/ contents with the version of this directory coming from clean Magento 2 site, and then keep applying patches until directories were the same. I was able to do this, it proved that about 25 of these patches were really applied.

Recently I wrote a blog post explaining how to deal with the second part of this challenge – check Rocket Web blog to see how to apply Magento 2 core patches.

Meet Magento 2014 Poland remarks

Magento, ,

It’s been almost two months since I got back from the Meet Magento Poland 2014 conference. These last two months were really busy for me and a lot of things are still going on. I haven’t got a chance to write about the event yet. I decided to not write a full story, but I’m going to wrap up a random list of notes and thoughts I have gathered after the conference.

My remarks

  • interesting case study of a multi-store and multi-language Magento integration was presented by LPP and Accenture
  • it was hard to switch between Business and Technology tracks and I lost more than half of Kuba Zwoliński presentation about iBeacon. I hope to see videos from the conference sometime soon!
  • there was a nice introduction to Magento 2 caching by Marko Martinović. I was surprised that the Magento 2 Community Edition includes Full Page Cache support
  • good points on contributing to open source Magento extensions and projects in general by Tsvetan Stoychev from Jarlssen. I still need to review extensions published on Jarlssen github
  • I couldn’t watch Damian Luszczymuk’s presentation about Docker as I chose the Flexible Billing talk by our CEO, Matt MacDougall. However I was really lucky and Damian showed me the presentation in brief later that day. I promise to dig into Docker soon.
  • there was an interesting case study on Magento-SAP integration made for Mennica Polska by Robert Żochowski from Bold Agency. I’m excited to buy some gold from a store based on Magento.
  • I got some interesting ideas about introducing developers to Magento development and noted a bunch of training materials from Ben Marks presentation
  • Daniel Sloof made a good overview on HHVM and I’m curious to see Magento 2 running on that engine
  • our booth with space figures and NASA suits was very successful, bringing a lot of people to our place!
  • I missed the presentation about Magento indexers by Maciej Ostrowski and I hope to see the video soon
  • I saw a cool demo of cobby.io, tool which allows me to manage product data in Excel. It sounds like a crazy idea but looks really interesting and works nice
  • community dinner at Podwale 25 restaurant was delicious. I hope to visit that cool place next time I’m in Warsaw
  • enjoyed Thomas Goletz’s story about Gobi desert race and about Chinese Magento branch
  • conference iPhone application was really cool, however iBeacon stuff didn’t work on my iPhone. It’s time to move to the new one

These are just random thoughts I got after the conference. I really liked the event and I hope to go to Meet Magento 2015 later this year :-)

Photo made by Viacheslav Kravchuk, Atwix. Thanks!

How to prepare Magento 2 beta package for offline use

Magento,

Magento 2 comes with a composer installer and all external dependencies including sample data are being installed using composer. However, I needed to have a simple way to install Magento 2 along with sample data in an offline environment, without using composer. I had a few reasons to do this – I wanted to have a fast way to install Magento 2 multiple times and I wanted to test command line installing for MageTesting.com purposes.

Main goals are:

  • avoid downloading more than 1 GB of data each time
  • let it work in offline mode
  • operate with smaller packages
  • simplify steps needed to install Magento 2

Cloning GIT repository and downloading dependencies resulted in downloading more than 1GB of data:

  • Magento 2 code cloned with packages downloaded using composer: 471.3 MB (194 MB after gzipping)
  • sample data media: 590.9 MB (zipped)
  • sample data code: 0.2 MB (zipped)

I decided to prepare a Magento 2 package which contains only code needed to run application and to prepare sample data package which could be installed just by copy pasting that into Magento 2. Recently I was playing with a sample data compression script provided by Vinai Kopp, and I made a fork which can compress Magento 2 sample data.

At the end I have the following packages:

  • Magento 2 code (26 MB, gzipped)
  • compressed sample data media (92MB, zipped)
  • sample data code: 0.2 MB (zipped)

I know there is a composer cache. I know I could use Vagrant/Docker or other virtualization, but still I wanted to avoid overcomplicating the process. If you see that use case useful, please find all needed steps described below.

Just keep in mind it is written for 0.42.0-beta1 release of Magento 2 and it is not a recommended way to install Magento 2.

Prepare Magento 2 package

1. Clone GIT repository

git clone git@github.com:magento/magento2.git

2. Install composer dependencies

composer.phar update

3. Remove huge directories not needed to run application

rm -rf dev/tests
rm -rf .git
rm -rf vendor/magento/zendframework1/documentation
rm -rf vendor/magento/zendframework1/tests
rm -rf vendor/magento/zendframework1/demos

4. Prepare package

tar czf magento2-0.42.0-beta1.tar.gz -C magento2/ .

Prepare Magento 2 sample data package

1. Downlod demo data

curl -O http://packages.magento.com/_packages/magento_sample-data-0.42.0-beta1.zip
curl -O http://packages.magento.com/_packages/magento_sample-data-media-0.42.0-beta1.zip

2. Compress demo data

compress-sample-data-magento2.sh magento_sample-data-media-0.42.0-beta1.zip

Install Magento 2 using created package

1. Prepare directory and unpack package:

mkdir magento2
tar xzf magento2-0.42.0-beta1.tar.gz -C magento2/

2. Set required permissions

cd magento2
chmod -R 777 var/
chmod -R 777 pub/media/
chmod -R 777 pub/static
chmod -R 777 app/etc/

3. Run Setup

php -f setup/index.php install 
    --base_url=http://local.magento2new.com/ 
    --backend_frontname=admin 
    --db_host=127.0.0.1 
    --db_name=mage2 
    --db_user=mage2 
    --db_pass=mage2 
    --admin_firstname=John 
    --admin_lastname=Doe 
    --admin_email=john@example.com 
    --admin_username=admin 
    --admin_password=admin 
    --language=en_US 
    --currency=USD 
    --timezone=Europe/Warsaw

Install sample data using package

1. Unpack media sample data

unzip -q -d pub/media/ compressed-magento_sample-data-media-0.42.0-beta1.zip

2. Unpack sample data code

mkdir dev/tools/Magento/Tools/SampleData
unzip -q -d dev/tools/Magento/Tools/SampleData magento_sample-data-0.42.0-beta1.zip

3. Install sample data

php -f dev/tools/Magento/Tools/SampleData/install.php -- --admin_username=admin

4. Make sure newly added files are writable:

chmod -R 777 pub/media/
chmod -R 777 pub/static

This one is a little dirty, but as far as I know composer doesn’t support installing local packages.

Let me know if you find this article useful and if you have any thoughts around that.

How to easily dump Magento database with n98-magerun

Magento, , ,

It looks I felt in love with a n98-magerun tool. I already talked about my favourite n98-magerun commands and about a command which allows to generate fake customer data. Today I’m going to continue n98-magerun post series and focus on a command which allows to make database dump very easily. n98-magerun.phar db:dump in addition to commands mentioned in previous blog posts is another one must have.

The command allows to dump a database very easily. Similar to a n98magerun mysql-client command, it doesn’t require me to enter a password and look for any connection details. It automatically generates a dump file name based on a current date and time, allows to use built-in filters to exclude big utility tables and finally creates an archive after completing the dump.

Sample calls may look as follows:

n98-magerun.phar db:dump
n98-magerun.phar db:dump --strip "@stripped @ee_changelog @idx"
n98-magerun.phar db:dump --strip "@stripped @ee_changelog @idx" --compression=gz

First call produced a 5.1 GB file containing all database tables. Second stripped a bunch of database tables such as changelog tables, index tables, reports or logs tables and it resulted in reducing the database dump to 3.1 GB. Third one called with a compression option reduced size to about 251 MB. Third gain is expected and it could be also achieved with one additional gz command call, but it’s really convenient to do a gzipped dump in one command call.

Stripping not needed database tables can save gigabytes of transfer when working with a big database. However, it won’t fit all use cases, for example in which you need to prepare a dump to debug index problems where you need to get an exact state of a Magento application database.

Get a free Meet Magento PL 2014 ticket!

Magento,

It’s only 9 days left to a Meet Magento PL 2014 conference which will take place on November 24th and 25th in Warsaw. There is a lot of interesting presentations announced for this event in the Meet Magento PL agenda. Rocket Web Inc. will also be on a stage – Matt MacDougall will talk about Flexible Billing models.

Are you going to attend to a conference and you haven’t got your ticket yet? Or maybe you are going to bring a friend? I have one free ticket to give away! If you want to get one, just write a comment under the post and say why do you want to go to the conference.

Contest begins on November 15th and ends on November 19th. I will announce the person who gets the pass on November 20th.

See you in Warsaw, make sure to stop at our booth :-)

Bring a lot of customers to Magento store… using n98-magerun

Magento, ,

Recently I talked about my favourite n98-magerun commands, this time I’ll continue that topic. Let’s see how to easily add a lot of customers to a Magento store. Of course it’s about adding fake data for testing purposes, unfortunately n98-magerun doesn’t help in bringing real ones :-)

Command customer:create:dummy allows you to create dummy customer accounts for testing purposes. It supports a few different locales and allows to generate any amount of accounts.

$ n98-magerun.phar customer:create:dummy 1000 pl_PL
[…]
Please select a website: 1

It took about 13 minutes to create 1000 customers, so it is able to fill a database with thousands of customers in a few hours. Sample output looks as follows:

Customer wieczorek.julia.152196@example.org with password QAELhMP! successfully created
Customer fabian.291162@example.net with password 8grCBZ9$ successfully created
Customer emil.108200@example.com with password ARWRJALy successfully created
[…]

Internally it uses https://github.com/fzaninotto/Faker. That library allows it to generate other assets like texts (either lorem ipsum or more real ones), addresses, phone numbers, companies, date and times, internet accounts, payment data and a few others.

I would love to see the ability to generate products using Faker and have a few more dummy methods incorporated into n98-magerun. The ability to generate products, customer addresses, maybe sales data would be great. However, the ability to generate fake customer accounts helps a lot and it’s a great start.

Top 10 n98-magerun command calls

Magento, ,

n98-magerun is a very useful tool when working on Magento stores. It saves time and makes a lot of Magento development related tasks much easier. Check out a list of my 10 favorite commands.

1. dev:module:rewrite:conflicts

This one is a must have. It’s a good replacement for an Extension Conflict module which I have been using before I found the n98-magerun. It shows rewrite conflicts in a clean ASCII table:

n98-magerun.phar dev:module:rewrite:conflicts

2. mysql-client

Command opens a mysql command line client without need of looking for a host, a port and credentials in local.xml file.

n98-magerun.phar mysql-client

However, it doesn’t work on a server with proc_open() function disabled.

3. cache:flush

It flushes all Magento caches. If a store uses filesystem cache, this command isn’t much more helpful than simple rm -rf var/cache* var/full_page_cache/* . However, when using the n98-magerun I don’t need to wonder what cache type is used in a store and this command provides an interface working for all cache types.

n98-magerun.phar cache:flush

The only glitch is that in some cases it doesn’t clean cache if file permissions are not correct.

4. sys:cron:list

Prints list of all active CRON jobs configured in a Magento store. It shows output in readable way, showing a job code, minutes, hours, days, month and week days.

n98-magerun.phar sys:cron:list

Another usable command related to CRON may be sys:cron:history which lists recently finished CRON jobs.

5. sys:setup:run

It’s useful to run database updates from a command line when installing an extension or upgrading Magento.

n98-magerun.phar sys:setup:run

Recently I used that when working on a Magento upgrade from 1.12 to 1.14. Frontend kept throwing error as PHP code was trying to use things which haven’t existed in the database yet, but sys:setup:run command dealt with that perfectly.

6. dev:theme:duplicates

This could be used when working on templates, to clean them from not needed files. I ran that command in 3 projects I’m working on and it found a few duplicated templates.

n98-magerun.phar dev:theme:duplicates enterprise/rocketweb base/default

7. sys:info

Command allows to quickly show an overall picture on the store. It shows a store edition and version, list of vendors coming from all code pools and a few other information. Additionally it shows basic factors which can determine the store size – amount of attributes, categories and products.

n98-magerun.phar sys:info

It’s handy when starting to work on an already existing site.

8. sys:check

It checks for missing system paths such as media/, var/ or a local.xml config file and checks if required PHP modules are installed. For example on my local server it keeps complaining about a missing index.php.sample file and about missing bytecode cache extension. I will need to get rid of this to get a nice green output :-)

n98-magerun.phar sys:check

Additionally it checks if each base URL contains dot, however I don’t get this part.

9. dev:console

This opens an interactive console with Magento initialized. It’s marked as experimental but works pretty well. It allows to run code in interactive way, most useful when I need to check multiple objects one by one eg. go through blog posts. For example to open console run:

n98-magerun.phar dev:console

And then run code to dump the blog post data:

Zend_Debug::dump(Mage::getModel('blog/post')->load(3)->debug());
Zend_Debug::dump(Mage::getModel('blog/post')->load(4)->debug());
Zend_Debug::dump(Mage::getModel('blog/post')->load(5)->debug());

Or order:

Zend_Debug::dump(Mage::getModel('sales/order')->load(109)->debug());

10. sys:modules:list

It lists all modules along with fields like code pool, key, version and status.

n98-magerun.phar sys:modules:list

Documentation says it’s possible to filter list by a code pool and a status, but I couldn’t get it working. It simply shows nothing when I use any filter. However, even without filtering it’s pretty useful.

I chose these 10 commands as most interesting, but there are many other handy tools: a command for working with database dumps, a command for debugging Magento configs and all toggle commands which allow to enable/disable things like a cache, a demo notice, template hints and many more.

Take a look on http://magerun.net to see all of them or better just play with the tool. Also make sure to subscribe my RSS feed or my twitter as soon I will talk more about cool n98-magerun use cases.