Tools for non-programmers to manage websites are growing in demand. No-code solutions increase bandwidth. Many SaaS products promise marketers and project managers a way to get things done without a developer. A visual website editor baked into an app is an important technology for digital product builders. The ability for a software service to interact with a client’s website can be a critical selling point.
SplitWit A/B Testing is an example of a web app that had that requirement. It lets user specify an existing website and displays it in an editor window. The user can then click on elements, make changes, and add new content. Those changes are actually applied to the live website by a JavaScript snippet. This post explains how it was built using PHP and jQuery.
Showing the website via iFrame
Displaying an already existing website on your app is easy with an iFrame. But, since users should be able to view and edit any page they choose, challenges arise. If a non-SSL URL is used (http:// instead of https://), the editor app will throw a “mixed-content warning”. If the page has X-FRAME-OPTIONS set to DENY, then the request will fail. To get around these issues I load an internal page into the iFrame that contains all of the DOM code from the one specified.
var pageIframe = $('<iframe id="page-iframe" style="" src="/page-to-edit.php?baseUrl='+baseUrl+'&url=<?php echo $url; ?>"></iframe>').appendTo(".page-iframe-wrapper");
The page to be edited is entered into an input field. That 3rd party webpage’s URL is passed along to my internal “page-to-edit.php” as a query parameter. From there, I get its full code using PHP Simple HTML DOM Parser. You’ll notice that I also pass along a “base url” – which is the root domain of the page. I’m able to grab it using JavaScript.
<?php
$url = "";
if(isset($_GET['url']) && strlen($_GET['url']) > 0){
$url = $_GET['url'];
}
?>
function removeSubdirectoryFromUrlString(url, ssl){
var ssl = ssl || false;
if(url.indexOf("https://")){
ssl = true;
}
url = url.replace("http://", "");
url = url.replace("https://", "");
var pathArray = url.split("/")
url = pathArray[0];
if(ssl){
url = "https://" + url;
}else{
url = "http://" + url;
}
return url;
}
var url = "<?php echo $url; ?>";
var ssl = false;
var pageIframe;
if(url.length > 0){
if(url.indexOf("https:") !== -1){
ssl = true
}
var baseUrl = removeSubdirectoryFromUrlString(url, ssl);
if(baseUrl.slice(-1) !== "/"){
baseUrl = baseUrl + "/";
}
}
I need to append that to any assets (images, scripts, etc.) that use relative references – or else they won’t load.
<?php
// Report all errors
error_reporting(E_ALL);
ini_set("display_errors", 1);
require 'simple_html_dom.php';
class HtmlDomParser {
/**
* @return \simplehtmldom_1_5\simple_html_dom
*/
static public function file_get_html() {
return call_user_func_array ( '\simplehtmldom_1_5\file_get_html' , func_get_args() );
}
/**
* get html dom from string
* @return \simplehtmldom_1_5\simple_html_dom
*/
static public function str_get_html() {
return call_user_func_array ( '\simplehtmldom_1_5\str_get_html' , func_get_args() );
}
}
$base_url = $_GET['baseUrl'];
$url = $_GET['url'];
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
$html = curl_exec($ch);
$redirectedUrl = curl_getinfo($ch, CURLINFO_EFFECTIVE_URL);
$url = $redirectedUrl;
$parse = parse_url($url);
$base_url = $parse["scheme"] . "://" . $parse["host"];
$html = @HtmlDomParser::file_get_html($url);
if(substr($base_url, 0, 1) == "/"){
$base_url = substr_replace($base_url ,"",-1);
}
if($html === FALSE) {
echo "Sorry, we do not have permission to analyze that website. ";
return;
}
foreach($html->find('script') as $element){
$src = $element->src;
// echo "<script>console.log('starting src: ".$src."')</script>";
if (strlen($src) > 0 && strpos($src, '//') === false){
if(substr($src, 0, 1) !== "/"){
$src = "/" . $src;
}
$element->src = $base_url . $src;
}
if(strlen($element->integrity) > 0){
$element->integrity = "";
}
// echo "<script>console.log('final src: ".$base_url . $src."')</script>";
// echo $element->src . "\n";
}
foreach($html->find('link') as $element){
$src = $element->href;
if (strlen($src) > 0 && strpos($src, '//') === false){
if(substr($src, 0, 1) !== "/"){
$src = "/" . $src;
}
$element->href = $base_url . $src;
}
if(strlen($element->integrity) > 0){
$element->integrity = "";
}
}
foreach($html->find('a') as $element){
$src = $element->href;
if (strlen($src) > 0 && strpos($src, '//') === false){
if(substr($src, 0, 1) !== "/"){
$src = "/" . $src;
}
$element->href = $base_url . $src;
}
}
foreach($html->find('img') as $element){
$src = $element->src;
if (strlen($src) > 0 && strpos($src, '//') === false){
if(substr($src, 0, 1) !== "/"){
$src = "/" . $src;
}
$element->src = $base_url . $src;
}
}
foreach($html->find('source') as $element){
$src = $element->srcset;
$sources = explode(",",$src);
$src = trim($sources[0]);
if (strlen($src) > 0 && strpos($src, '//') === false){
if(substr($src, 0, 1) !== "/"){
$src = "/" . $src;
}
$element->srcset = $base_url . $src;
}
}
echo $html;
?>
I check against five different element types for assets that could need to be updated: <script>, <link>, <a>, <img>, and <source>. In version 1.0, I had missed the <link> and <source> elements.
Now that I was able to load any website into my editor UI, I had to think about layout. I split the screen in half, having the right side display the website, and the left side showing editor options.
Update: Eventually, I noticed that some pages were not loading properly. I found it to be two issues: a security error & a permission-denied response. The security error was a PHP problem. The permission-denied (401) response was due to server settings. Some sites don’t want to be accessed by bots and try to ensure that an actual user is making the request. I was able to fix it by passing context settings to the ‘file_get_html’ method.
$arrContextOptions=array(
"ssl"=>array(
"verify_peer"=>false,
"verify_peer_name"=>false,
),
"http" => array(
"header" => "User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36"
)
);
$html = file_get_html($url, false, stream_context_create($arrContextOptions));
Element selection
Page-clicks in the editor window need to be intercepted to stop links from being followed and any other actions from happening. Instead, any thing clicked is selected by the editor and its details shown in the options panel.
As elements are moused over, a semi-transparent red highlight is applied as a visual cue. It is achieved by injecting CSS into the iFrame and adding mouseenter and mouseout event listeners.
pageIframe.on('load', function(){
var style = "<style>.highlighted{background-color:rgba(255, 0, 0, 0.5);} </style>";
pageIframe.contents().find("body").prepend(style);
pageIframe.contents().find("body *").mouseenter(function(){
$(this).addClass('highlighted');
}).mouseout(function(){
$(this).removeClass('highlighted');
});
});
When something is clicked, normal functionality is stopped by using the web browser’s preventDefault() method. Once the element is grabbed (using the this keyword), we want to be sure that we’re as deep into the DOM tree as possible. That way, our “Text / HTML” content won’t contain unnecessary code tags.
To do so, we use a while loop to iterate an arbitrary number of times. Eight iterations seems to be the number that gets us to the heart of any selected content, without taking too much time.
First, we check if the element has any child nodes – if it does, we grab the first one. If that node happens to be a <style> or <script> tag, we move onto the second instead. As long as that new node is not a formatting element (<strong>, <em>, etc.), we set it as the new element before continuing our loop.
testSelectorEl = $(this);
var i = 8;
while(i > 0){
if ( $(testSelectorEl).children().length > 0 ) {
nextEl = $(testSelectorEl).find(">:first-child");
if(nextEl.is( "style" ) || nextEl.is( "script" ) || nextEl.is( "noscript" )){
nextEl = $(testSelectorEl).find(">:nth-child(2)");
}
if ( !nextEl.is( "u" ) && !nextEl.is( "i" ) && !nextEl.is( "strong" ) && !nextEl.is( "em" )) {
testSelectorEl = nextEl;
}
}
i--;
}
The above code is added to the click event listener on all elements within the iFrame. Next, below that code, we determine a unique CSS selector by which we can later reference the element. This is important for writing any changes to the JavaScript snippet to effect changes on the 3rd-party website.
var node = testSelectorEl;
var path = "";
while (node.length) {
var realNode = node[0], name = realNode.localName;
if (!name) break;
name = name.toLowerCase();
var parent = node.parent();
var siblings = parent.children(name);
if (siblings.length > 1) {
name += ':eq(' + siblings.index(realNode) + ')';
}
path = name + (path ? '>' + path : '');
node = parent;
}
var value = path;
$(".selector-input").val(value);
The final iFrame on-load function looks like this:
var url = "<?php echo $url; ?>";
var ssl = false;
var pageIframe;
if(url.length > 0){
if(url.indexOf("https:") !== -1){
ssl = true
}
var baseUrl = removeSubdirectoryFromUrlString(url, ssl);
if(baseUrl.slice(-1) !== "/"){
baseUrl = baseUrl + "/";
}
var style = "<style>.highlighted{background-color:rgba(255, 0, 0, 0.5);} </style>";
var pageIframe = $('<iframe id="page-iframe" style="" src="/new-page-to-edit.php?baseUrl='+baseUrl+'&url=<?php echo $url; ?>"></iframe>').appendTo(".page-iframe-wrapper");
pageIframe.on('load', function(){
pageIframe.contents().find("body").prepend(style);
pageIframe.contents().find("body *").mouseenter(function(){
$(this).addClass('highlighted');
testSelectorEl = $(this);
}).mouseout(function(){
$(this).removeClass('highlighted');
}).click(function(e){
e.preventDefault();
e.stopPropagation();
//dig deeper down the dom
var i = 8;
while(i > 0){
if ( $(testSelectorEl).children().length > 0 ) {
nextEl = $(testSelectorEl).find(">:first-child");
if(nextEl.is( "style" ) || nextEl.is( "script" ) || nextEl.is( "noscript" )){
nextEl = $(testSelectorEl).find(">:nth-child(2)");
}
if ( !nextEl.is( "u" ) && !nextEl.is( "i" ) && !nextEl.is( "strong" ) && !nextEl.is( "em" )) {
testSelectorEl = nextEl;
}
}
i--;
}
var node = testSelectorEl;
var path = "";
while (node.length) {
var realNode = node[0], name = realNode.localName;
if (!name) break;
name = name.toLowerCase();
var parent = node.parent();
var siblings = parent.children(name);
if (siblings.length > 1) {
name += ':eq(' + siblings.index(realNode) + ')';
}
path = name + (path ? '>' + path : '');
node = parent;
}
var value = path;
$(".selector-input").val(value); //for html insert section (redundant for change element section)
if(! $(".insert-html-wrap").is(':visible')){
selectNewElement(value); // prepare editor options
$(".page-editor-info").offset().top; //scroll user to selector input
}
return false;
});
//make sure images load
pageIframe.contents().find("img").each(function(){
var src = $(this).attr("src");
if(src && src.length > 0 && src.indexOf("//") == -1){ //if not absolute reference
if(src.charAt(0) !== "/"){
src = "/" + src;
}
$(this).attr("src", baseUrl + src);
}
});
//make sure links aren't followed
pageIframe.contents().find("a").each(function(){
var href = $(this).attr("href");
$(this).attr("href", "");
$(this).attr("data-href", href);
});
pageIframe.contents().find("body").attr("style", "cursor: pointer !important");
$(".loading-iframe").hide();
}); //page-iframe load
}else{
//no URL found
$(".loading-iframe").hide();
}
Lastly, we prepare the editor options panel. This involves setting the original content for the selected element and removing any newly added features (modals, sticky-bars) that were not saved.
Editor Options
After an element is clicked its content and style properties are loaded into the editor panel. The original values are stored in memory. As users edit values a change indicator icon is revealed and the right-panel editor view is updated in real time.
var testSelectorEl;
var testSelectorElPath = "";
var testSelectorElHtml = "";
var testSelectorElImage = "";
var testSelectorElLink = "";
var originalVisibilityState = "";
var originalValues = [];
originalValues['height'] = "";
originalValues['width'] = "";
originalValues['margin'] = "";
originalValues['padding'] = "";
originalValues['border'] = "";
originalValues['font-family'] = "";
originalValues['font-weight'] = "";
originalValues['font-style']= "";
originalValues['text-decoration'] = "";
originalValues['background'] = "";
originalValues['css'] = "";
originalValues['classes'] = "";
$(".html-input").keyup(function(){
var value = $(this).val();
if (value !== testSelectorElHtml){
$(this).parent().find(".change-indicator").show();
}else{
$(this).parent().find(".change-indicator").hide();
}
if($(".change-indicator").is(":visible")){
$(".element-change-save-btn").removeAttr("disabled");
}else{
$(".element-change-save-btn").attr("disabled", "disabled");
}
var selector = $(".selector-input").val();
var iFrameDOM = $("iframe#page-iframe").contents()
iFrameDOM.find(selector).html(value);
});
Besides changing existing elements, new ones can also be added. The “insert content” section, also based on selecting an element, lets users add new text, html, or images before or after whatever has been clicked. In both sections, the CSS selector can also be manually typed (instead of clicking). Adding or editing images is handled by a custom built image upload gallery that leverages AWS S3 and PHP.
Out-of-the-box elements, such as sticky bars and modals, can also be added with a few clicks and configurations. The HTML and CSS for those are pre-built, with variables for any options that may be set. Any changes made are saved to the database in relation to the user account, project, experiment, and variation.
function addSticky(){
$conn = $this->connection;
$accountid = $this->accountid;
$variationid = $_GET['variationid'];
$experimentid = $_GET['experimentid'];
$text = $_POST['text'];
$color = $_POST['color'];
$background = $_POST['background'];
$position = $_POST['position'];
$linkurl = $_POST['linkurl'];
$insertStatement = "INSERT INTO `variationchange` (accountid, variationid, experimentid, selector, changetype, changecode, content) VALUES (:accountid, :variationid, :experimentid, :selector, :changetype, :changecode, :content)";
$changetype = "stickybar";
$selector = "body";
$link_opening = "";
$link_closing = "";
if(strlen($linkurl) > 0){
$link_opening = "<a style='color:".$color."' href='".$linkurl."'>";
$link_closing = "</a>";
}
$sticky_html = "<div style='font-weight:bold;".$position.":0;position:fixed;z-index:100000;left:0px;text-align:center;padding:8px 20px;width:100%;background:".$background.";color:".$color."' id='splitwit-sticky'><p style='margin:0px'>".$link_opening.$text.$link_closing."</p></div>";
$changecode = '$("body").append("'.$sticky_html.'")';
$stmt = $conn->prepare($insertStatement);
$stmt->bindParam(':accountid', $accountid);
$stmt->bindParam(':variationid', $variationid);
$stmt->bindParam(':experimentid', $experimentid);
$stmt->bindParam(':selector', $selector);
$stmt->bindParam(':changetype', $changetype);
$stmt->bindParam(':changecode', $changecode);
$stmt->bindParam(':content', $text);
$stmt->execute();
$this->writeSnippetFile($variationid);
}
Writing changes to the snippet
Every project has a unique JavaScript file that users must add to their webpage. The file is hosted by SplitWit, so website owners only need to copy/paste a snippet. The WordPress and Shopify plugins automatically add the snippet, making it even more friendly to non-developers.
Each project may contain multiple experiments with their own changes, metrics, variations, and conditions. After the data is produced by a SQL join statement, it is massaged into a nested object and parsed in JSON. That JavaScript output is concatenated to the necessary libraries and helper functions.
function writeSnippetFile($variationid=false, $experimentid=false, $projectid=false){
$conn = $this->connection;
$variationid = $variationid || false;
$experimentid = $experimentid || false;
if(isset($_GET['variationid'])){
$variationid = $_GET['variationid'];
}
if(isset($_GET['experimentid'])){
$experimentid = $_GET['experimentid'];
}
if($variationid){
$variationid = $_GET['variationid'];
$sql = "SELECT experiment.projectid FROM `variation` right join `experiment` on variation.experimentid = experiment.experimentid WHERE variationid = ?";
$result = $conn->prepare($sql);
$result->execute(array($variationid));
$experiment_row = $result->fetch(PDO::FETCH_ASSOC);
}elseif($experimentid){
$experimentid = $_GET['experimentid'];
$sql = "SELECT projectid FROM `experiment` WHERE experimentid = ?";
$result = $conn->prepare($sql);
$result->execute(array($experimentid));
$experiment_row = $result->fetch(PDO::FETCH_ASSOC);
}
if(!$projectid){
$projectid = $experiment_row['projectid'];
}
$sql = "SELECT experiment.experimentid, experiment.status, experimentcondition.experimentconditionid, variation.variationid, variationchange.variationchangeid, variationchange.changecode, variation.css, variation.javascript, experimentcondition.url, experimentcondition.matchtype, experimentcondition.conditiontype, metric.metricid, metric.type, metric.input, metric.urlmatch FROM `experiment` left join `metric` on metric.experimentid = experiment.experimentid join `experimentcondition` on experimentcondition.experimentid = experiment.experimentid join `variation` on variation.experimentid = experiment.experimentid left join `variationchange` on variationchange.variationid = variation.variationid WHERE experiment.projectid = ?";
// echo "<br />"."variationid: ".$variationid . "<br />";
// echo "<br />"."experimentid: ".$experimentid . "<br />";
// echo "<br />"."projectid: ".$projectid . "<br />";
// $this->status = "SQL: " . $sql;
$result = $conn->prepare($sql);
$result->execute(array($projectid));
$experiment_row = $result->fetchAll(PDO::FETCH_ASSOC);
//turn flat array, into a nested one
$endResult = array();
foreach($experiment_row as $row){
if (!isset($endResult[$row['experimentid']])){
$endResult[$row['experimentid']] = array(
'experimentid' => $row['experimentid'],
'status' => $row['status'],
'conditions' => array(),
'variations' => array(),
'metrics' => array()
);
}
if (!isset($endResult[$row['experimentid']]['conditions'][$row['experimentconditionid']])){
$endResult[$row['experimentid']]['conditions'][$row['experimentconditionid']] = array(
'experimentconditionid' => $row['experimentconditionid'],
'url' => $row['url'],
'matchtype' => $row['matchtype'],
'conditiontype' => $row['conditiontype']
);
}
if (!isset($endResult[$row['experimentid']]['variations'][$row['variationid']])){
$endResult[$row['experimentid']]['variations'][$row['variationid']] = array(
'variationid' => $row['variationid'],
'javascript' => $row['javascript'],
'css' => $row['css'],
'changes' => array()
);
}
if (!isset($endResult[$row['experimentid']]['variations'][$row['variationid']]['changes'][$row['variationchangeid']])){
$endResult[$row['experimentid']]['variations'][$row['variationid']]['changes'][$row['variationchangeid']] = array(
'variationchangeid' => $row['variationchangeid'],
'changecode' => $row['changecode']
);
}
if (!isset($endResult[$row['experimentid']]['metrics'][$row['metricid']])){
$endResult[$row['experimentid']]['metrics'][$row['metricid']] = array(
'metricid' => $row['metricid'],
'type' => $row['type'],
'input' => $row['input'],
'urlmatch' => $row['urlmatch']
);
}
}
$json_output = json_encode($endResult);
// echo $json_output;
// get snippet file name.
$sql = "SELECT snippet from `project` where projectid=?";
$result = $conn->prepare($sql);
$result->execute(array($projectid));
$projectrow = $result->fetch(PDO::FETCH_ASSOC);
$filename = $projectrow['snippet'];
// echo $filename;
$snippet_template = file_get_contents("/var/www/html/snippet/snippet-template.min.js");
// concat json to snippet template. write file
$myfile = fopen("/var/www/html/snippet/".$filename, "w") or die("Unable to open file!");
//if there are any animation changes, include the necessary library.
$txt = "";
if(strpos($json_output, "addClass('animated") !== false){
$txt .= 'var head=document.getElementsByTagName("head")[0],link=document.createElement("link");link.rel="stylesheet";link.type="text/css";link.href="https://www.splitwit.com/css/animate.min.css";link.media="all";head.appendChild(link); ';
}
$txt .= "window.splitWitExperiments = ".$json_output . "\n" . $snippet_template;
fwrite($myfile, $txt) or die("Unable to save file!");
fclose($myfile);
//if this project is for a shopify app, we need to update the snippet cache
$update_snippet_url = "https://www.splitwit.com/service-layer/shopify-app-service.php?method=updateSnippetScriptTag&projectid=".$projectid;
$params = [];
$curl_update_snippet = $this->curlApiUrl($update_snippet_url, $params);
}
One of the helper functions is a method that checks if the current URL matches the experiment’s conditions to run on:
function testUrl(testurl, conditions){
if(testurl.search(/(?:[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?\.)+[a-z0-9][a-z0-9-]{0,61}[a-z0-9]/) < 0){
console.log("no domain")
return window.inputError($(".test-url-input"), "Please test a valid URL.");
}
var valid = false;
for (i = 0; i < conditions.length; i++) {
var url = conditions[i].url;
var matchtype = conditions[i].matchtype;
var conditiontype = conditions[i].conditiontype;
//exact match
if(matchtype == "exact" && conditiontype == "target" && url == testurl){
valid = true;
}
if(matchtype == "exact" && conditiontype == "exclude" && url == testurl){
valid = false;
}
//basic match
if(matchtype == "basic"){
//strip off querystring, hashtags, protocols, and www from url and testurl - then compare
var cleanTestUrl = testurl.toLowerCase();
var cleanUrl = url.toLowerCase();
if (cleanTestUrl.indexOf("?") > 0) {
cleanTestUrl = cleanTestUrl.substring(0, cleanTestUrl.indexOf("?"));
}
if (cleanUrl.indexOf("?") > 0) {
cleanUrl = cleanUrl.substring(0, cleanUrl.indexOf("?"));
}
if (cleanTestUrl.indexOf("&") > 0) {
cleanTestUrl = cleanTestUrl.substring(0, cleanTestUrl.indexOf("&"));
}
if (cleanUrl.indexOf("&") > 0) {
cleanUrl = cleanUrl.substring(0, cleanUrl.indexOf("&"));
}
if (cleanTestUrl.indexOf("#") > 0) {
cleanTestUrl = cleanTestUrl.substring(0, cleanTestUrl.indexOf("#"));
}
if (cleanUrl.indexOf("#") > 0) {
cleanUrl = cleanUrl.substring(0, cleanUrl.indexOf("#"));
}
cleanTestUrl = cleanTestUrl.replace(/^(?:https?:\/\/)?(?:www\.)?/i, "");
cleanUrl = cleanUrl.replace(/^(?:https?:\/\/)?(?:www\.)?/i, "");
cleanTestUrl = cleanTestUrl.replace(/\/$/, "");
cleanUrl = cleanUrl.replace(/\/$/, ""); //remove trailing slash
// console.log(cleanTestUrl);
// console.log(cleanUrl);
if(conditiontype == "target" && cleanUrl == cleanTestUrl){
valid = true;
}
if(conditiontype == "exclude" && cleanUrl == cleanTestUrl){
valid = false;
}
}
//substring match
if(matchtype == "substring"){
if(testurl.includes(url) && conditiontype == "target"){
valid = true;
}
if(testurl.includes(url) && conditiontype == "exclude"){
valid = false;
}
}
} //end conditions loop
$(".test-url-msg").hide();
if(valid){
$(".valid-msg").fadeIn();
}else{
$(".invalid-msg").fadeIn();
}
}
While the snippet code is specific to this use-case (experimental A/B UI changes), the visual editor can be used in a variety of other contexts. You can look through more of the code I used in this GitHub repository I created.