PHP Web Scraper : Extract XML Sitemap

Assalamualaikom w.b.t and Greetings,

Today i just want to share the simple code to extract sitemap.xml and obtain all the titles and publish date. This code not use any extension like simplexml but just basic regex to extract certain part of the xmls.

Here the codes:

<?

echo "welcome to the ExtractSitemapXml PHP Project!";

$url ='http://myskali.blogspot.com/sitemap.xml';


$result = file_get_contents($url);

$pat = "~<loc>(.*?)<\/loc>\s*<lastmod>(.*?)<\/lastmod>~is";
preg_match_all($pat, $result, $match);

echo '<pre>';

print_r(array_slice($match,1)); //slice index 0 all pattern match

?>

You can test the code as live here:

Simple Sitemap XML Grabber and Extractor

SKALI SPIKE UP PROGRAMME

Assalamualaikom w.b.t and Greetings,

Well guy an girls, we are in the edge of our last month to work at SKALI as Young Skalian under Spike Up or SPARK programs. I felt time was run out fast and until now i am still not enough to get a full knowledge here.

Its almost 5 month and i am not sure either to continue the contract or just to find another company. I like and love to be Skalian here, lots of friend and great environment too. Just i don't like several political stuff that feel he or she is better and other are always lazy and doing mistakes.

For next batch, you are welcome here. This post i just want to write in Malay Language from this point.

Kepada kekawan seperjuangan semua, terima kasih di atas sokongan, kemesraan dan tunjuk ajar yang bermakna untuk saya.

Kepada bakal-bakal Spiker atau Skali Muda, tahniah sekiranya anda terpilih untuk menyertai program ini. Di sini ada banyak gudang ilmu yang berguna untuk anda.

Program SKALI SPIKE UP juga dikenali sebagai SPIKER dalam SKALI GROUP dan didalam pejabat nanti kita akan digelar sebagai Young SKALI atau pekerja Skali Muda. Ala macam Imam Muda tu.

Apa itu program SPIKE nie?

Program nie bertujuan menyediakan latihan berkaitan Teknologi IT dalam suasana pekerjaan yang sebenar di pejabat SKALI atau mana-mana subsidari nya.

Maksudnya anda akan diberikan suasana perkerjaan dan disamping itu anda akan diberikan latihan dan kelas berjadual supaya kemahiran anda dalam sesuatu bidang tu bertambah.

Dulu-dulu program ini mamakan masa 3 tahun, kemudian 1 tahun dan sekarang dipendekkan sehingga menjadi 6 bulan sahaja secara kontrak. Saya pula berada dalam batch pertama yang kena 6 bulan bermula 2 Julai 2012 yang lalu.

Tempat Kerja?

Biasanya anda akan ditempat di Plaza Sentul atau Menara Sentul Jalan Sentul Pasar Kuala Lumpur. Ada juga yang ditempatkan di IDC Serdang untuk menjaga server di sana.

Sekarang IDC Serdang dah berpindah ke Cyberjaya. Jadi kemungkinan anda akan ditempatkan disana. Ala, paling busuk pun di Plaza Sentul, ibu pejabat Alam Teknokrat Sdn Bhd (SKALI).

Suasana di Plaza Sentul?

Wow, masjid ada di sebelah. Ada pasar raya di tingkat bawah plaza dan rumah sewa yang banyak di sekitarnya sehingga ke Batu Caves. Kedai makan dengan harga berpatutan juga ada. Nasi dgn lauk dalam RM4.00 sehingga RM4.50. Berbaloi-baloi.

Gaji atau Elauan?

Alhamdulillah, anda akan diberi gaji sebanyak RM1500 sebulan secara kasar. Batch-batch yang lepas mereka dapat RM2000 sebulan tau.

Kemudahan lain?

KWSP (EPF), SOCSO, Insurance Coverage, Red Card (perubatan) dan cuti tahunan berjumlah 7 hari.

Bilik Pentri, surat khabar, bilik mandi dengan pemanas air, tandas yang cantik dan makmal komputer serba lengkap.

Apa yang anda dapat nanti?

Setiap spiker atau young Skalian akan mendapat satu set komputer peribadi (PC) untuk kegunaan kerja. Anda akan ada  ruang meja sendiri dan kerusi ofis yang cantik.

Yang Bagusnyer?

Ada ramai kawan dan staf-staf yang baik-baik belaka. Mereka juga ada sebahagiannya bekas SPIKER seperi En. Faiz bin Saari, En. Sariman, Cik Rajwant Kaur a/p Visan Singh, Cik Ruhaida Othman, En. Fadhil Omar, Chee Lee Chin (CK Tan), dan ramai lagi. Jadi sekiranya prestasi anda bagus anda mungkin akan diserap dalam SKALI.

Masa Kerja?

Isnin sehingga Jumaat. Dari 9 pagi sehingga 5.30 petang. Sabtu dan Ahad anda boleh rehat dan buat amal. Kalau ada masa kerja tambahan anggaplah anda mendapat ilmu lebih.

Program yang Ada ialah:

1. Project Management, (jaga projek2 SKALI berada di bawah SWS)

2. Content Management, (kemaskini portal laman web, tambah dan upload kandungan)

3. Web Design, (Reka grafik photoshop, html, php dreamweaver)

4. .NET programmer, (application implementation, asp)

5. JAVA programmer, (Liferay, Coding portal guna java)

6. Technical Writing, (Arkitektur kanduan laman, bahasa inggeris, IA)

7. Software Quality Analysis, (kena belajar koding untuk semak perisian, portal, Quality Assurance)

8. Network Administration (repair, maintenance, trobleshoot pcs, network device, server, software dan hardware)　

9. Junior Technical Trainers. (pengurusan kelas, pengajaran, tutor, kedatangan, impor export pengajar, susun kelas)



Dan sebenarnya ada beberapa lagi tetapi saya dah tak ingat.

Sesiapa yang ada pertanyaan boleh meletakkan komen di bawah InsyaAllah saya akan cuba menjawabnya.

Sekian,
Mike2012

Tambahan 11 Disember 2013 - Spike 2013 adalah yang terakhir. Rujuk berikut Spike Up 2014?

PHP Web Scraper : Extract RSS Title

Lets Grab SkaliCloud RSS

Assalamualaikom w.b.t and Greetings,

Today i will show you how to grab RSS feed and parse its titles. The function and flow is:

1. File get contents to grab the Rss page
2. Regex to extract the desired text

Next, file_get_contents can handle remote file access. Even its slow i hope you can learn it as beginner. Next time i will show you how to use my favorite CURL.

Back to the tutorial. Here how we can grab RSS:

$result = file_get_contents("http://www.skalicloud.com/category/blog/feed/");

Then the $result will contain XML containing all post information like title, date of post, author and etc..

Here how rss xml looks like:

<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
    xmlns:content="http://purl.org/rss/1.0/modules/content/"
    xmlns:wfw="http://wellformedweb.org/CommentAPI/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:atom="http://www.w3.org/2005/Atom"
    xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
    xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
    >

<channel>
    <title>SKALI Cloud : Malaysia First True Public Cloud Service &#187; Blog</title>
    <atom:link href="http://www.skalicloud.com/category/blog/feed/" rel="self" type="application/rss+xml" />
    <link>http://www.skalicloud.com</link>
    <description>On-Demand Cloud Computing Highly Elastic</description>
    <lastBuildDate>Tue, 04 Oct 2011 01:53:46 +0000</lastBuildDate>
    <language>en</language>
    <sy:updatePeriod>hourly</sy:updatePeriod>
    <sy:updateFrequency>1</sy:updateFrequency>
    <generator>http://wordpress.org/?v=3.0.1</generator>
        <item>
        <title>Cloud vs Traditional Hosting</title>
        <link>http://www.skalicloud.com/blog/article/cloud-vs-traditional-hosting/</link>
        <comments>http://www.skalicloud.com/blog/article/cloud-vs-traditional-hosting/#comments</comments>
        <pubDate>Tue, 04 Oct 2011 01:53:14 +0000</pubDate>
        <dc:creator>admin</dc:creator>
                <category><![CDATA[Article]]></category>

        <guid isPermaLink="false">http://www.skalicloud.com/?p=1264</guid>
        <description><![CDATA[An article by Mr. Edward from Brilliant Thinking compared traditional hosting methods with cloud hosting. About 3 years ago, I<a href="http://www.skalicloud.com/blog/article/cloud-vs-traditional-hosting/">(more...)</a>]]></description>

Our objective is to get any title between <item> and </item> where all post information are located.

Also the problem is there are other <title> before <item> tag so we need to remove this portion of source to just starting from <item> element. But using array we can filter out unused array from matched Regex.

My best weapon is using Xpath for complex extraction but for beginner i will show you my 2nd weapon is uing REGEX. Regex is very good if we know how to pattern it.

Here my regex pattern:

~<title>(.*?)<\/title>~is //this will extract any chars from between any <title> tag

Ok here my full source code:

<?php
$url ='http://www.skalicloud.com/category/blog/feed/';


$result = file_get_contents($url);

$pat = "~<title>(.*?)<\/title>~is";
preg_match_all($pat, $result, $match);

echo '<pre>';

print_r($match[1]);
?>

And result:

Array
(
    [0] => SKALI Cloud : Malaysia First True Public Cloud Service » Blog
    [1] => Cloud vs Traditional Hosting
    [2] => When is the Public Cloud Most Useful for Enterprises?
    [3] => Planting the Seeds for Your Business Growth…
    [4] => Announcing a Community Discount Model of SKALI Cloud Services
    [5] => TV News Coverage of the SKALI-Microsoft Partnership
    [6] => PRESS RELEASE: SKALI Adds Microsoft Windows-based Hosting
    [7] => SKALI Cloud @ Intuit Technology Day 2011
    [8] => SKALI Cloud Capacity Increased, Serving More Clients
    [9] => Reseller Program Officially Launched at the NEF-Awani ICT Fair
    [10] => Securing with Virtual Private Cloud (VPC)
)

This how its looks like.

Ok. If you compare between the RSS and our result. 

You will notice array index zero (0) is not the item title. Its title of blog.

To remove index(0) i prefer using array_slice.

Here how:

<?php
$url ='http://www.skalicloud.com/category/blog/feed/';

 
$result = file_get_contents($url);

$pat = "~<title>(.*?)<\/title>~is";
preg_match_all($pat, $result, $match);

echo '<pre>';

print_r(array_slice($match[1],1)); //slice index 0
?>

And the result:

Array
(
    [0] => Cloud vs Traditional Hosting
    [1] => When is the Public Cloud Most Useful for Enterprises?
    [2] => Planting the Seeds for Your Business Growth…
    [3] => Announcing a Community Discount Model of SKALI Cloud Services
    [4] => TV News Coverage of the SKALI-Microsoft Partnership
    [5] => PRESS RELEASE: SKALI Adds Microsoft Windows-based Hosting
    [6] => SKALI Cloud @ Intuit Technology Day 2011
    [7] => SKALI Cloud Capacity Increased, Serving More Clients
    [8] => Reseller Program Officially Launched at the NEF-Awani ICT Fair
    [9] => Securing with Virtual Private Cloud (VPC)
) 

Its fun right?

You can test the code here:

SEE LIVE ACTION

Wahuallam... 

Convert Title Into Slug URL Using PHP

During my job while uploading content for MPS project, i am always facing the same problem. Its always make me become lazious to change all link title into slugify format manually. I am weird why this new portal system can't handle this conversion.

To overcome this issues, i've created a new php code to assist my work. This is very basic code based on Regex pattern to remove all non alphanumeric character and change all spaces into dashes.

Here is what i want for this code:

1. Remove non alphanumeric chars.
2. Change spaces into dashes.
3. All must lowercase.

To fulfill above rule here the simplest code:

function slugify($txt)
{
  // replace all non alphanumeric into dashes
  $text = preg_replace('/\W+/', '-', $txt);

  // trim and lowercase
  return strtolower(trim($txt, '-'));
}
?>

Example scenario:

Mistery of Mine & How to : Extracting IP Values IPv6 2012 under 10% value

Will converted into:

mistery-of-mine-how-to-extracting-ip-values-ipv6-2012-under-10-value

Hope it help others.

Php Web Scraper Its Hard?

Php Web Scraper

Assalamualaikom and Good Day everyone.

Did you know about PHP Web Scraper?

Web Scraper is my favorite technology involving PHP server coding. Its very interesting stuff to explore. Today i will explain a simplified overview of how data gets to and from the web browser to our php applications.

Web Scraping

The flow of process starting by copying any site html pages and then filtering or extracting with several php parsing techniques.

To grab any page we can use Curl (its my priority), HTTP stream socket, or others library like Snoopy. I always prefer using Curl due its user friendly functioning and faster. 

To extract any portion of html, text or any element of scraped site we need a better php libraries such as php SimpleDom, PHPQuery or as simple DomXpath. Using common ordinary regex and string manipulation will take lots of time to coded.

Next post i will show you several technique to using Curl as web grabber.

Good Luck!

Lesson To PHP Web Scraping