Tag Archives: xml_rpc

WordPress XML-RPC Endpoint Validator

WordPress uses the XML-RPC remote publishing interface in order to provide a standardized way to transfer data between 3rd party clients, like mobile/desktop apps, and the core of the CMS. These 3rd party clients may ask for something like an XML-RPC (or API) endpoint, which will usually is your blog domain followed by “xmlrpc.php". So, for example, if your site was at daniloercoli.wordpress.com, your endpoint would be:

https://daniloercoli.wordpress.com/xmlrpc.php or https://daniloercoli.wordpress.com/xmlrpc.php

The Endpoint requires username/password, this is the same username and password that you use to login to WordPress, and it’s secure like the browser login.

Themes and Plugins can interact and change the behavior of the XML-RPC Endpoint, by adding new methods, or modifying the response document. There are actually a lot of plugins that interact with it in some way, but unfortunately, plugins and themes could break the XML-RPC endpoint,  or invalidate the XML response document. When I need to reply to a question about the EndPoint that’s not working on a self-hosted installation of WordPress I usually reply with the following words:

Did you try with the default theme and with no active plugins?

Trying it with the default theme and plugins disabled will help pin down where the problem is. If everything works then you go back and enable your theme and then try it again. If everything still works then you go back and activate each plugin one at a time until you find the one that is causing the breakage.

The Validator
Since my team and I usually spend a lot of time replying to questions on issues on the XML-RPC endpoint of WordPress sites, I’ve decided to write a web tool that can be used to easily test the Endpoint. This tool should speed up our work, and will give to users a simple way to test the endpoint of their sites. I wrote this tool in PHP (https://github.com/daniloercoli/WordPress-XML-RPC-Validator), by using WordPress as framework, and for simplicity in this first release it’s a plugin that adds a page to your site. That page embeds the XML-RPC validator. It does some XML-RPC calls to your blog, checks the response documents, and in case of errors gives back hints on how to fix the issue. At this moment it’s just an experiment, no fancy UI there, but I will probably improve it over the time. A live version of the validator is available here.  Please, contribute to the project 🙂

 

Resources

Reducing the memory usage of the XML-RPC Endpoint in WordPress

WordPress uses the XML-RPC remote publishing interface in order to provide a standardized way to transfer data between a 3rd party client, like a mobile app, and the CMS. Nowadays many popular clients use this capability, useful for transferring posts, pages, and comments, but unfortunately it requires a lot of memory on the server side if used to publish picture or video. I wanted to reduce the memory  usage during the parsing process of  XML-RPC requests in WordPress, and make the upload of big pictures and videos available on the XML-RPC Endpoint of cheap shared hosting installations with strict php memory limit.

A quick recap. WordPress uses a conventional approach to XML-RPC request parsing, it relies on the basic idea that the whole XML-RPC request is loaded into memory ($HTTP_RAW_POST_DATA), and it often stores intermediate values into memory. Since a single XML-RPC request document might have a large picture or a video in it, the parsing process could take a lot of memory. For instance, when you upload a picture of 3Mb, you may need to upload at least 4Mb of data, due to the base64 encoding process that will increase the image size by approximately 35%. Hence the full parsing procedure requires at least 7Mb of memory: a 4Mb variable that holds the XML-RPC document and 3Mb for the variable that holds the data of the image. That’s not good. If you want upload a short video of 100Mb, recorded on your mobile device, you may have to set a really high value for the ‘memory_limit’ option, that could not be possible on shared hosting.

That said, I though the XML-RPC parsing process could be improved by using an XML Streaming parser, that reads the data directly from the input stream, and stores intermediate values on disk. In details I thought to get rid of $HTTP_RAW_POST_DATA, read the request from the input stream php://, parse it by using a Streaming Parser, store intermediate values on disk, and pack these changes into a plugin.

Unfortunately I got through a closed road, since PHP always populates the variable $HTTP_RAW_POST_DATA on POST requests with “text/xml” content-type. Even if you set the directive always_populate_raw_post_data to Off,  PHP populates that variable. The only exception are requests with content-type of “application/x-www-form-urlencoded” or “multipart/form-data”. I investigated more, and seems that there isn’t a simple way to get rid of  $HTTP_RAW_POST_DATA without modifying the PHP code. Shared hosting installation could not get huge benefits from this plugin, since they can’t install a modified version of PHP, anyway it could help a little bit, because it doesn’t store intermediate value in memory but uses the disk space.

The plugins does the following:

  • Gets the php input stream (php://input)
  • Reads from the input stream, and parse the content by chunck (Get rid of $HTTP_RAW_POST_DATA).
  • Doesn’t store any partial values, or the final parsed value in memory. It uses a tmp file in the tmp directory (Only for base64 data).
  • Changes the function mw_newMediaObject to use a path to the input file on disk rather accept the whole content as parameter.
  • Introduces a new function that copy the uploaded file to the right location.

If you want to test it here is the link to the code. (Note: This plugin is only an ‘alpha’ version, do not use in production). I don’t think I will develop it further, since a REST(ful) API, was already published on WordPress.com, and I hope it will be soon adopted on core too.

Special thanks to Luca Ercoli that assisted me with the php core stuff.