Create an account


Welcome, Guest
You have to register before you can post on our site.

Username
  

Password
  





Search Forums

(Advanced Search)

Forum Statistics
» Members: 19,400
» Latest member: tobiasz
» Forum threads: 21,157
» Forum posts: 21,725

Full Statistics

Online Users
There are currently 251 online users.
» 1 Member(s) | 246 Guest(s)
Applebot, Bing, Google, Yandex, SickProdigy

 
  Open Liberty Java runtime now available to Red Hat Runtimes subscribers
Posted by: xSicKxBot - 12-02-2025, 10:14 AM - Forum: Java Language, JVM, and the JRE - No Replies

Open Liberty Java runtime now available to Red Hat Runtimes subscribers

Open Liberty is a lightweight, production-ready Java runtime for containerizing and deploying microservices to the cloud, and is now available as part of a Red Hat Runtimes subscription. If you are a Red Hat Runtimes subscriber, you can write your Eclipse MicroProfile and Jakarta EE apps on Open Liberty and then run them in containers on Red Hat OpenShift, with commercial support from Red Hat and IBM.

Develop cloud-native Java microservices


Open Liberty is designed to provide a smooth developer experience with a one-second startup time, a low memory footprint, and our new dev mode:

Tweet about Open Liberty Dev Mode.

Open Liberty provides a full implementation of MicroProfile 3 and Jakarta EE 8. MicroProfile is a collaborative project between multiple vendors (including Red Hat and IBM) and the Java community that aims to optimize enterprise Java for writing microservices. With a four-week release schedule, Liberty usually has the latest MicroProfile release available soon after the spec is published.

Also, Open Liberty is supported in common developer tools, including VS Code, Eclipse, Maven, and Gradle. Server configuration (e.g., adding or removing a capability, or “feature,” to your app) is through an XML file. Open Liberty’s zero migration policy means that you can focus on what’s important (writing your app!) and not have to worry about APIs changing under you.

Deploy in containers to any cloud


When you’re ready to deploy your app, you can just containerize it and deploy it to OpenShift. The zero migration principle means that new versions of Open Liberty features will not break your app, and you can control which version of the feature your app uses.

Monitoring live microservices is enabled by MicroProfile Metrics, Health, and OpenTracing, which add observability to your apps. The emitted metrics from your apps and from the Open Liberty runtime can be consolidated using Prometheus and presented in Grafana.

Learn with the Open Liberty developer guides


Our Open Liberty developer guides are available with runnable code and explanations to help you learn how to write microservices with MicroProfile and Jakarta EE, and then to deploy them to Red Hat OpenShift.

Get started


To get started with Open Liberty, try the Packaging and deploying applications guide and the Deploying microservices to OpenShift guide.

Share

The post Open Liberty Java runtime now available to Red Hat Runtimes subscribers appeared first on Red Hat Developer.



https://www.sickgaming.net/blog/2019/11/...bscribers/

Print this item

  [Tut] Python BS4 – How to Scrape Absolute URL Instead of Relative Path
Posted by: xSicKxBot - 12-02-2025, 10:14 AM - Forum: Python - No Replies

[Tut] Python BS4 – How to Scrape Absolute URL Instead of Relative Path

5/5 – (1 vote)

Summary: Use urllib.parse.urljoin() to scrape the base URL and the relative path and join them to extract the complete/absolute URL. You can also concatenate the base URL and the absolute path to derive the absolute path; but make sure to take care of erroneous situations like extra forward-slash in this case.

Quick Answer


When web scraping with BeautifulSoup in Python, you may encounter relative URLs (e.g., /page2.html) instead of absolute URLs (e.g., http://example.com/page2.html). To convert relative URLs to absolute URLs, you can use the urljoin() function from the urllib.parse module.

Below is an example of how to extract absolute URLs from the a tags on a webpage using BeautifulSoup and urljoin:


from bs4 import BeautifulSoup
import requests
from urllib.parse import urljoin # URL of the webpage you want to scrape
url = 'http://example.com' # Send an HTTP request to the URL
response = requests.get(url)
response.raise_for_status() # Raise an error for bad responses # Parse the webpage content
soup = BeautifulSoup(response.text, 'html.parser') # Find all the 'a' tags on the webpage
for a_tag in soup.find_all('a'): # Get the href attribute from the 'a' tag href = a_tag.get('href') # Use urljoin to convert the relative URL to an absolute URL absolute_url = urljoin(url, href) # Print the absolute URL print(absolute_url)

In this example:

  • url is the URL of the webpage you want to scrape.
  • response is the HTTP response obtained by sending an HTTP GET request to the URL.
  • soup is a BeautifulSoup object that contains the parsed HTML content of the webpage.
  • soup.find_all('a') finds all the a tags on the webpage.
  • a_tag.get('href') gets the href attribute from an a tag, which is the relative URL.
  • urljoin(url, href) converts the relative URL to an absolute URL by joining it with the base URL.
  • absolute_url is the absolute URL, which is printed to the console.

Now that you have a quick overview let’s dive into the specific problem more deeply and discuss various methods to solve this easily and effectively. ?

Problem Formulation


Problem: How do you extract all the absolute URLs from an HTML page?

Example: Consider the following webpage which has numerous links:


Now, when you try to scrape the links as highlighted above, you find that only the relative links/paths are extracted instead of the entire absolute path. Let us have a look at the code given below, which demonstrates what happens when you try to extract the 'href' elements normally.

from bs4 import BeautifulSoup
import urllib.request
from urllib.parse import urljoin
import requests web_url = 'https://sayonshubham.github.io/'
headers = {"User-Agent": "Mozilla/5.0 (CrKey armv7l 1.5.16041) AppleWebKit/537.36 (KHTML, like Gecko) " "Chrome/31.0.1650.0 Safari/537.36"}
# get() Request
response = requests.get(web_url, headers=headers)
# Store the webpage contents
webpage = response.content
# Check Status Code (Optional)
# print(response.status_code)
# Create a BeautifulSoup object out of the webpage content
soup = BeautifulSoup(webpage, "html.parser")
for i in soup.find_all('nav'): for url in i.find_all('a'): print(url['href'])

Output:

/
/about
/blog
/finxter
/

The above output is not what you desired. You wanted to extract the absolute paths as shown below:

https://sayonshubham.github.io/
https://sayonshubham.github.io/about
https://sayonshubham.github.io/blog
https://sayonshubham.github.io/finxter
https://sayonshubham.github.io/

Without further delay, let us go ahead and try to extract the absolute paths instead of the relative paths.

Method 1: Using urllib.parse.urljoin()


The easiest solution to our problem is to use the urllib.parse.urljoin() method.

According to the Python documentation: urllib.parse.urljoin() is used to construct a full/absolute URL by combining the “base URL” with another URL. The advantage of using the urljoin() is that it properly resolves the relative path, whether BASE_URL is the domain of the URL, or the absolute URL of the webpage.

from urllib.parse import urljoin URL_1 = 'http://www.example.com'
URL_2 = 'http://www.example.com/something/index.html' print(urljoin(URL_1, '/demo'))
print(urljoin(URL_2, '/demo'))

Output:

http://www.example.com/demo
http://www.example.com/demo

Now that we have an idea about urljoin, let us have a look at the following code which successfully resolves our problem and helps us to extract the complete/absolute paths from the HTML page.

Solution:

from bs4 import BeautifulSoup
import urllib.request
from urllib.parse import urljoin
import requests web_url = 'https://sayonshubham.github.io/'
headers = {"User-Agent": "Mozilla/5.0 (CrKey armv7l 1.5.16041) AppleWebKit/537.36 (KHTML, like Gecko) " "Chrome/31.0.1650.0 Safari/537.36"}
# get() Request
response = requests.get(web_url, headers=headers)
# Store the webpage contents
webpage = response.content
# Check Status Code (Optional)
# print(response.status_code)
# Create a BeautifulSoup object out of the webpage content
soup = BeautifulSoup(webpage, "html.parser")
for i in soup.find_all('nav'): for url in i.find_all('a'): print(urljoin(web_url, url.get('href')))

Output:

https://sayonshubham.github.io/
https://sayonshubham.github.io/about
https://sayonshubham.github.io/blog
https://sayonshubham.github.io/finxter
https://sayonshubham.github.io/

Method 2: Concatenate The Base URL And Relative URL Manually


Another workaround to our problem is to concatenate the base part of the URL and the relative URLs manually, just like two ordinary strings. The problem, in this case, is that manually adding the strings might lead to “one-off” errors — try to spot the extra front slash characters / below:

URL_1 = 'http://www.example.com/'
print(URL_1+'/demo') # Output --> http://www.example.com//demo

Therefore to ensure proper concatenation, you have to modify your code accordingly such that any extra character that might lead to errors is removed. Let us have a look at the following code that helps us to concatenate the base and the relative paths without the presence of any extra forward slash.

Solution:

from bs4 import BeautifulSoup
import urllib.request
from urllib.parse import urljoin
import requests web_url = 'https://sayonshubham.github.io/'
headers = {"User-Agent": "Mozilla/5.0 (CrKey armv7l 1.5.16041) AppleWebKit/537.36 (KHTML, like Gecko) " "Chrome/31.0.1650.0 Safari/537.36"}
# get() Request
response = requests.get(web_url, headers=headers)
# Store the webpage contents
webpage = response.content
# Check Status Code (Optional)
# print(response.status_code)
# Create a BeautifulSoup object out of the webpage content
soup = BeautifulSoup(webpage, "html.parser")
for i in soup.find_all('nav'): for url in i.find_all('a'): # extract the href string x = url['href'] # remove the extra forward-slash if present if x[0] == '/': print(web_url + x[1:]) else: print(web_url+x)

Output:

https://sayonshubham.github.io/
https://sayonshubham.github.io/about
https://sayonshubham.github.io/blog
https://sayonshubham.github.io/finxter
https://sayonshubham.github.io/

⚠ Caution: This is not the recommended way of extracting the absolute path from a given HTML page. In situations when you have an automated script that needs to resolve a URL but at the time of writing the script you don’t know what website your script is visiting, in that case, this method won’t serve your purpose, and your go-to method would be to use urlljoin. Nevertheless, this method deserves to be mentioned because, in our case, it successfully serves the purpose and helps us to extract the absolute URLs.

Conclusion


In this article, we learned how to extract the absolute links from a given HTML page using BeautifulSoup. If you want to master the concepts of Pythons BeautifulSoup library and dive deep into the concepts along with examples and video lessons, please have a look at the following link and follow the articles one by one wherein you will find every aspect of BeautifulSoup explained in great details.

YouTube Video

? Recommended: Web Scraping With BeautifulSoup In Python

With that, we come to the end of this tutorial! Please stay tuned and subscribe for more interesting content in the future.

The post Python BS4 – How to Scrape Absolute URL Instead of Relative Path appeared first on Be on the Right Side of Change.



https://www.sickgaming.net/blog/2023/09/...tive-path/

Print this item

  [Tut] AJAX File Upload with Progress Bar using JavaScript
Posted by: xSicKxBot - 12-02-2025, 10:13 AM - Forum: PHP Development - No Replies

[Tut] AJAX File Upload with Progress Bar using JavaScript

by Vincy. Last modified on June 30th, 2023.

If you want to upload a file using AJAX and also need to show a progress bar during the upload, you have landed on the right page.

This article has an example code for JavaScript AJAX file upload with a progress bar.

An AJAX-based file upload is a repeatedly needed requirement for a web application.

It is for providing an inline editing feature with the uploaded file content. For example, the following tasks can be achieved using the AJAX file upload method.

  1. Photo or banner update on the profile page.
  2. Import CSV or Excel files to load content to the data tables.

View demo
ajax file upload with progress bar javascript

HTML upload form


This HTML shows the input to choose a file. This form has a button that maps its click event with an AJAX handler.

In a previous tutorial, we have seen a jQuery example for uploading form data with a chosen file binary.

But in this example, the HTML doesn’t have any form container. Instead, the form data is created by JavaScript before processing the AJAX.

This HTML has a container to show the progress bar. Once the progress is 100% complete, a success message is added to the UI without page refresh.

<div class="phppot-container tile-container text-center"> <h2>AJAX File Upload with Progress Bar using JavaScript</h2> <input type="file" id="fileUpload" /> <br> <br> <button on‌click="uploadFile()">Upload</button> <div class="progress"> <div class="progress-bar" id="progressBar"></div> </div> <br> <div id="uploadStatus"></div>
</div>

AJAX file upload request with progress bar


This section is the core of this example code. This example’s HTML and PHP files are prevalent, as seen in other file upload examples.

The script below follows the steps to achieve the AJAX file upload.

  1. It reads the file binary chosen in the file input field.
  2. It instantiates JavaScript FormData and appends the file binary into it.
  3. It creates an XMLHttpRequest handle.
  4. This handle uses the ‘upload’ property to get XMLHttpRequestUpload object.
  5. This XMLHttpRequestUpload object tracks the upload progress in percentage.
  6. It creates event listeners to update the progressing percentage and the upload status.
  7. Then finally, it posts the file to the PHP endpoint like usual AJAX programming.
function uploadFile() { var fileInput = document.getElementById('fileUpload'); var file = fileInput.files[0]; if (file) { var formData = new FormData(); formData.append('file', file); var xhr = new XMLHttpRequest(); xhr.upload.addEventListener('progress', function (event) { if (event.lengthComputable) { var percent = Math.round((event.loaded / event.total) * 100); var progressBar = document.getElementById('progressBar'); progressBar.style.width = percent + '%'; progressBar.innerHTML = percent + '%'; } }); xhr.addEventListener('load', function (event) { var uploadStatus = document.getElementById('uploadStatus'); uploadStatus.innerHTML = event.target.responseText; }); xhr.open('POST', 'upload.php', true); xhr.send(formData); }
}

PHP endpoint to move the uploaded file into a directory


This PHP  has a standard code to store the uploaded file in a folder using the PHP move_uploaded_file(). The link has the code if you want to store the uploaded file and save the path to the database.

This endpoint creates a unique name for the filename before upload. It is a good programming practice, but the code will work without it, also.

It is for stopping file overwriting in case of uploading different files in the same name.

Note: Create a folder named “uploads” in the project root. Give sufficient write permissions.

<?php if ($_SERVER['REQUEST_METHOD'] === 'POST' && isset($_FILES['file'])) { $file = $_FILES['file']; // file will be uploaded to the following folder // you should give sufficient file permissions $uploadDir = 'uploads/'; // unique file name generated $fileName = uniqid() . '_' . $file['name']; // moving the uploaded file from temp location to our target location if (move_uploaded_file($file['tmp_name'], $uploadDir . $fileName)) { echo 'File uploaded successfully.'; } else { echo 'Failed to upload file.'; }
}

View demo Download

↑ Back to Top



https://www.sickgaming.net/blog/2023/06/...avascript/

Print this item

  Quarkus: Modernize “helloworld” JBoss EAP quickstart, Part 1
Posted by: xSicKxBot - 12-01-2025, 05:47 PM - Forum: Java Language, JVM, and the JRE - No Replies

Quarkus: Modernize “helloworld” JBoss EAP quickstart, Part 1

Quarkus is, in its own words, “Supersonic subatomic Java” and a “Kubernetes native Java stack tailored for GraalVM & OpenJDK HotSpot, crafted from the best of breed Java libraries and standards.” For the purpose of illustrating how to modernize an existing Java application to Quarkus, I will use the Red Hat JBoss Enterprise Application Platform (JBoss EAP) quickstarts helloworld quickstart as sample of a Java application builds using technologies (CDI and Servlet 3) supported in Quarkus.

It’s important to note that both Quarkus and JBoss EAP rely on providing developers with tools based—as much as possible—on standards. If your application is not already running on JBoss EAP, there’s no problem. You can migrate it from your current application server to JBoss EAP using the Red Hat Application Migration Toolkit. After that, the final and working modernized version of the code is available in the https://github.com/mrizzi/jboss-eap-quickstarts/tree/quarkus repository inside the helloworld module.

This article is based on the guides Quarkus provides, mainly Creating Your First Application and Building a Native Executable.

Get the code


To start, clone the JBoss EAP quickstarts repository locally, running:

$ git clone https://github.com/jboss-developer/jboss...starts.git Cloning into 'jboss-eap-quickstarts'... remote: Enumerating objects: 148133, done. remote: Total 148133 (delta 0), reused 0 (delta 0), pack-reused 148133 Receiving objects: 100% (148133/148133), 59.90 MiB | 7.62 MiB/s, done. Resolving deltas: 100% (66476/66476), done. $ cd jboss-eap-quickstarts/helloworld/

Try plain, vanilla helloworld


The name of the quickstart is a strong clue about what this application does, but let’s follow a scientific approach in modernizing this code, so first things first: Try the application as it is.

Deploy helloworld


  1. Open a terminal and navigate to the root of the JBoss EAP directory EAP_HOME (which you can download).
  2. Start the JBoss EAP server with the default profile by typing the following command:
$ EAP_HOME/bin/standalone.sh 

Note: For Windows, use the EAP_HOME\bin\standalone.bat script.

After a few seconds, the log should look like:

[org.jboss.as] (Controller Boot Thread) WFLYSRV0025: JBoss EAP 7.2.0.GA (WildFly Core 6.0.11.Final-redhat-00001) started in 3315ms - Started 306 of 527 services (321 services are lazy, passive or on-demand)
  1. Open http://127.0.0.1:8080 in a browser, and a page like Figure 1 should appear:
The JBoss EAP home page.

Figure 1: The JBoss EAP home page.

  1. Following instructions from Build and Deploy the Quickstart, deploy the helloworld quickstart and execute (from the project root directory) the command:
$ mvn clean install wildfly:deploy 

This command should end successfully with a log like this:

[INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 8.224 s 

The helloworld application has now been deployed for the first time in JBoss EAP in about eight seconds.

Test helloworld


Following the Access the Application guide, open http://127.0.0.1:8080/helloworld in the browser and see the application page, as shown in Figure 2:

JBoss EAP's Hello World.

Figure 2: JBoss EAP’s Hello World.

Make changes


Change the createHelloMessage(String name) input parameter from World to Marco (my ego is cheap):

writer.println("<h1>" + helloService.createHelloMessage("Marco") + "</h1>");

Execute again the command:

$ mvn clean install wildfly:deploy 

and then refresh the web page in the browser to check the message displayed changes, as shown in Figure 3:

JBoss EAP's Hello Marco.

Figure 3: JBoss EAP’s Hello Marco.

Undeploy helloworld and shut down


If you want to undeploy (optional) the application before shutting down JBoss EAP, run the following command:

$ mvn clean install wildfly:undeploy 

To shut down the JBoss EAP instance, enter Ctrl+C in the terminal where it’s running.

Let’s modernize helloworld


Now we can leave the original helloworld behind and update it.

Create a new branch


Create a new working branch once the quickstart project finishes executing:

$ git checkout -b quarkus 7.2.0.GA 

Change the pom.xml file


The time has come to start changing the application. starting from the pom.xml file. From the helloworld folder, run the following command to let Quarkus add XML blocks:

$ mvn io.quarkus:quarkus-maven-plugin:0.23.2:create 

This article uses the 0.23.2 version. To know which is the latest version is, please refer to https://github.com/quarkusio/quarkus/releases/latest/, since the Quarkus release cycles are short.

This command changed the pom.xml, file adding:

  • The property <quarkus.version> to define the Quarkus version to be used.
  • The <dependencyManagement> block to import the Quarkus bill of materials (BOM). In this way, there’s no need to add the version to each Quarkus dependency.
  • The quarkus-maven-plugin plugin responsible for packaging the application, and also providing the development mode.
  • The native profile to create application native executables.

Further changes required to pom.xml, to be done manually:

  1. Move the <groupId> tag outside of the <parent> block, and above the <artifactId> tag. Because we remove the <parent> block in the next step, the <groupId> must be preserved.
  2. Remove the <parent> block: The application doesn’t need the JBoss parent pom anymore to run with Quarkus.
  3. Add the <version> tag (below the <artifactId> tag) with the value you prefer.
  4. Remove the <packaging> tag: The application won’t be a WAR anymore, but a plain JAR.
  5. Change the following dependencies:
    1. Replace the javax.enterprise:cdi-api dependency with io.quarkus:quarkus-arc, removing <scope>provided</scope> because—as stated in the documentation—this Quarkus extension provides the CDI dependency injection.
    2. Replace the org.jboss.spec.javax.servlet:jboss-servlet-api_4.0_spec dependency with io.quarkus:quarkus-undertow, removing the <scope>provided</scope>, because—again as stated in the documentation—this is the Quarkus extension that provides support for servlets.
    3. Remove the org.jboss.spec.javax.annotation:jboss-annotations-api_1.3_spec dependency because it’s coming with the previously changed dependencies.

The pom.xml file’s fully changed version is available at https://github.com/mrizzi/jboss-eap-quickstarts/blob/quarkus/helloworld/pom.xml.

Note that the above mvn io.quarkus:quarkus-maven-plugin:0.23.2:create command, besides the changes to the pom.xml file, added components to the project. The added file and folders are:

  • The files mvnw and mvnw.cmd, and .mvn folder: The Maven Wrapper allows you to run Maven projects with a specific version of Maven without requiring that you install that specific Maven version.
  • The docker folder (in src/main/): This folder contains example Dockerfile files for both native and jvm modes (together with a .dockerignore file).
  • The resources folder (in src/main/): This folder contains an empty application.properties file and the sample Quarkus landing page index.html (more in the section “Run the modernized helloworld“).

Run helloworld


To test the application, use quarkus:dev, which runs Quarkus in development mode (more details on Development Mode here).

Note: We expect this step to fail as changes are still required to the application, as detailed in this section.

Now run the command to check if and how it works:

$ ./mvnw compile quarkus:dev [INFO] Scanning for projects... [INFO] [INFO] ----------------< org.jboss.eap.quickstarts:helloworld >---------------- [INFO] Building Quickstart: helloworld quarkus [INFO] --------------------------------[ war ]--------------------------------- [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ helloworld --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 2 resources [INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ helloworld --- [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- quarkus-maven-plugin:0.23.2:dev (default-cli) @ helloworld --- Listening for transport dt_socket at address: 5005 INFO [io.qua.dep.QuarkusAugmentor] Beginning quarkus augmentation INFO [org.jbo.threads] JBoss Threads version 3.0.0.Final ERROR [io.qua.dev.DevModeMain] Failed to start quarkus: java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors [error]: Build step io.quarkus.arc.deployment.ArcProcessor#validate threw an exception: javax.enterprise.inject.spi.DeploymentException: javax.enterprise.inject.UnsatisfiedResolutionException: Unsatisfied dependency for type org.jboss.as.quickstarts.helloworld.HelloService and qualifiers [@Default] - java member: org.jboss.as.quickstarts.helloworld.HelloWorldServlet#helloService - declared on CLASS bean [types=[javax.servlet.ServletConfig, java.io.Serializable, org.jboss.as.quickstarts.helloworld.HelloWorldServlet, javax.servlet.GenericServlet, javax.servlet.Servlet, java.lang.Object, javax.servlet.http.HttpServlet], qualifiers=[@Default, @Any], target=org.jboss.as.quickstarts.helloworld.HelloWorldServlet] at io.quarkus.arc.processor.BeanDeployment.processErrors(BeanDeployment.java:841) at io.quarkus.arc.processor.BeanDeployment.init(BeanDeployment.java:214) at io.quarkus.arc.processor.BeanProcessor.initialize(BeanProcessor.java:106) at io.quarkus.arc.deployment.ArcProcessor.validate(ArcProcessor.java:249) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at io.quarkus.deployment.ExtensionLoader$1.execute(ExtensionLoader.java:780) at io.quarkus.builder.BuildContext.run(BuildContext.java:415) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:2011) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1535) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1426) at java.lang.Thread.run(Thread.java:748) at org.jboss.threads.JBossThread.run(JBossThread.java:479) Caused by: javax.enterprise.inject.UnsatisfiedResolutionException: Unsatisfied dependency for type org.jboss.as.quickstarts.helloworld.HelloService and qualifiers [@Default] - java member: org.jboss.as.quickstarts.helloworld.HelloWorldServlet#helloService - declared on CLASS bean [types=[javax.servlet.ServletConfig, java.io.Serializable, org.jboss.as.quickstarts.helloworld.HelloWorldServlet, javax.servlet.GenericServlet, javax.servlet.Servlet, java.lang.Object, javax.servlet.http.HttpServlet], qualifiers=[@Default, @Any], target=org.jboss.as.quickstarts.helloworld.HelloWorldServlet] at io.quarkus.arc.processor.Beans.resolveInjectionPoint(Beans.java:428) at io.quarkus.arc.processor.BeanInfo.init(BeanInfo.java:371) at io.quarkus.arc.processor.BeanDeployment.init(BeanDeployment.java:206) ... 14 more 

It failed. Why? What happened?

The UnsatisfiedResolutionException exception refers to the HelloService class, which is a member of the HelloWorldServlet class (java member: org.jboss.as.quickstarts.helloworld.HelloWorldServlet#helloService). The problem is that HelloWorldServlet needs an injected instance of HelloService, but it can not be found (even if the two classes are in the very same package).

It’s time to return to Quarkus guides to leverage the documentation and understand how @Inject—and hence Contexts and Dependency Injection (CDI)—works in Quarkus, thanks to the Contexts and Dependency Injection guide. In the Bean Discovery paragraph, it says, “Bean classes that don’t have a bean defining annotation are not discovered.”

Looking at the HelloService class, it’s clear there’s no bean defining annotation, and one has to be added to have Quarkus to discover the bean. So, because it’s a stateless object, it’s safe to add the @ApplicationScoped annotation:

@ApplicationScoped public class HelloService { 

Note: The IDE should prompt you to add the required package shown here (add it manually if need be):

import javax.enterprise.context.ApplicationScoped; 

If you’re in doubt about which scope to apply when the original bean has no scope defined, please refer to the JSR 365: Contexts and Dependency Injection for Java 2.0—Default scope documentation.

Now, try again to run the application, executing again the ./mvnw compile quarkus:dev command:

$ ./mvnw compile quarkus:dev [INFO] Scanning for projects... [INFO] [INFO] ----------------< org.jboss.eap.quickstarts:helloworld >---------------- [INFO] Building Quickstart: helloworld quarkus [INFO] --------------------------------[ war ]--------------------------------- [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ helloworld --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 2 resources [INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ helloworld --- [INFO] Changes detected - recompiling the module! [INFO] Compiling 2 source files to /home/mrizzi/git/forked/jboss-eap-quickstarts/helloworld/target/classes [INFO] [INFO] --- quarkus-maven-plugin:0.23.2:dev (default-cli) @ helloworld --- Listening for transport dt_socket at address: 5005 INFO [io.qua.dep.QuarkusAugmentor] (main) Beginning quarkus augmentation INFO [io.qua.dep.QuarkusAugmentor] (main) Quarkus augmentation completed in 576ms INFO [io.quarkus] (main) Quarkus 0.23.2 started in 1.083s. Listening on: http://0.0.0.0:8080 INFO [io.quarkus] (main) Profile dev activated. Live Coding activated. INFO [io.quarkus] (main) Installed features: [cdi] 

This time the application runs successfully.

Run the modernized helloworld


As the terminal log suggests, open a browser to http://0.0.0.0:8080(the default Quarkus landing page), and the page shown in Figure 4 appears:

The Quarkus dev landing page.

Figure 4: The Quarkus dev landing page.

This application has the following context’s definition in the WebServlet annotation:

@WebServlet("/HelloWorld") public class HelloWorldServlet extends HttpServlet { 

Hence, you can browse to http://0.0.0.0:8080/HelloWorldto reach the page shown in Figure 5:

The Quarkus dev Hello World page.

Figure 5: The Quarkus dev Hello World page.

It works!

Make changes


Please, pay attention to the fact that the ./mvnw compile quarkus:dev command is still running, and we’re not going to stop it. Now, try to apply the same—very trivial—change to the code and see how Quarkus improves the developer experience:

writer.println("<h1>" + helloService.createHelloMessage("Marco") + "</h1>");

Save the file, and then refresh the web page to check that Hello Marco appears, as shown in Figure 6:

The Quarkus dev Hello Marco page.

Figure 6: The Quarkus dev Hello Marco page.

Take time to check the terminal output:

INFO [io.qua.dev] (vert.x-worker-thread-3) Changed source files detected, recompiling [/home/mrizzi/git/forked/jboss-eap-quickstarts/helloworld/src/main/java/org/jboss/as/quickstarts/helloworld/HelloWorldServlet.java] INFO [io.quarkus] (vert.x-worker-thread-3) Quarkus stopped in 0.003s INFO [io.qua.dep.QuarkusAugmentor] (vert.x-worker-thread-3) Beginning quarkus augmentation INFO [io.qua.dep.QuarkusAugmentor] (vert.x-worker-thread-3) Quarkus augmentation completed in 232ms INFO [io.quarkus] (vert.x-worker-thread-3) Quarkus 0.23.2 started in 0.257s. Listening on: http://0.0.0.0:8080 INFO [io.quarkus] (vert.x-worker-thread-3) Profile dev activated. Live Coding activated. INFO [io.quarkus] (vert.x-worker-thread-3) Installed features: [cdi] INFO [io.qua.dev] (vert.x-worker-thread-3) Hot replace total time: 0.371s 

Refreshing the page triggered the source code change detection and the Quarkus automagic “stop-and-start.” All of this executed in just 0.371 seconds (that’s part of the Quarkus “Supersonic Subatomic Java” experience).

Build the helloworld packaged JAR


Now that the code works as expected, it can be packaged using the command:

$ ./mvnw clean package

This command creates two JARs in the /target folder. The first is helloworld-<version>.jar, which is the standard artifact built from the Maven command with the project’s classes and resources. The second is helloworld-<version>-runner.jar, which is an executable JAR.

Please pay attention to the fact that this is not an uber-jar, because all of the dependencies are copied into the /target/lib folder (and not bundled within the JAR). Hence, to run this JAR in another location or host, both the JAR file and the libraries in the /lib folder have to be copied, considering that the Class-Path entry of the MANIFEST.MF file in the JAR explicitly lists the JARs from the lib folder.

To create an uber-jar application, please refer to the Uber-Jar Creation Quarkus guide.

Run the helloworld packaged JAR


Now, the packaged JAR can be executed using the standard java command:

$ java -jar ./target/helloworld-<version>-runner.jar INFO [io.quarkus] (main) Quarkus 0.23.2 started in 0.673s. Listening on: http://0.0.0.0:8080 INFO [io.quarkus] (main) Profile prod activated. INFO [io.quarkus] (main) Installed features: [cdi] 

As done above, open the http://0.0.0.0:8080 URL in a browser, and test that everything works.

Build the helloworld quickstart-native executable


So far so good. The helloworld quickstart ran as a standalone Java application using Quarkus dependencies, but more can be achieved by adding a further step to the modernization path: Build a native executable.

Install GraalVM


First of all, the tools for creating the native executable have to be installed:

  1. Download GraalVM 19.2.0.1 from https://github.com/oracle/graal/releases/tag/vm-19.2.0.1.
  2. Untar the file using the command:

$ tar xvzf graalvm-ce-linux-amd64-19.2.0.1.tar.gz

  1. Go to the untar folder.
  2. Execute the following to download and add the native image component:

$ ./bin/gu install native-image

  1. Set the GRAALVM_HOME environment variable to the folder created in step two, for example:

$ export GRAALVM_HOME={untar-folder}/graalvm-ce-19.2.0.1)

More details and install instructions for other operating systems are available in Building a Native Executable—Prerequisites Quarkus guide.

Build the helloworld native executable


As stated in the Building a Native Executable—Producing a native executable Quarkus guide, “Let’s now produce a native executable for our application. It improves the startup time of the application and produces a minimal disk footprint. The executable would have everything to run the application including the ‘JVM’ (shrunk to be just enough to run the application), and the application.”

To create the native executable, the Maven native profile has to be enabled by executing:

$ ./mvnw package -Pnative

The build took me about 1:10 minutes and the result is the helloworld-<version>-runner file in the /target folder.

Run the helloworld native executable


The /target/helloworld-<version>-runner file created in the previous step. It’s executable, so running it is easy:

$ ./target/helloworld-<version>-runner INFO [io.quarkus] (main) Quarkus 0.23.2 started in 0.006s. Listening on: http://0.0.0.0:8080 INFO [io.quarkus] (main) Profile prod activated. INFO [io.quarkus] (main) Installed features: [cdi] 

As done before, open the http://0.0.0.0:8080 URL in a browser and test that everything is working.

Next steps


I believe that this modernization, even of a basic application, is the right way to approach a brownfield application using technologies available in Quarkus. This way, you can start facing the issues and tackling them to understand and learn how to solve them.

In part two of this series, I’ll look at how to capture memory consumption data in order to evaluate performance improvements, which is a fundamental part of the modernization process.

Share

The post Quarkus: Modernize “helloworld” JBoss EAP quickstart, Part 1 appeared first on Red Hat Developer.



https://www.sickgaming.net/blog/2019/11/...rt-part-1/

Print this item

  [Tut] Python Int to String with Trailing Zeros
Posted by: xSicKxBot - 12-01-2025, 05:47 PM - Forum: Python - No Replies

[Tut] Python Int to String with Trailing Zeros

5/5 – (1 vote)

To add trailing zeros to a string up to a certain length in Python, convert the number to a string and use the ljust(width, '0') method. Call this method on the string, specifying the total desired width and the padding character '0'. This will append zeros to the right of the string until the specified width is achieved.

Challenge: Given an integer number. How to convert it to a string by adding trailing zeros so that the string has a fixed number of positions.

Example: For integer 42, you want to fill it up with trailing zeros to the following string with 5 characters: '42000'.

In all methods, we assume that the integer has less than 5 characters.

Method 1: string.ljust()


In Python, you can use the str.ljust() method to pad zeros (or any other character) to the right of a string. The ljust() method returns the string left-justified in a field of a given width, padded with a specified character (default is space).

Below is an example of how to use ljust() to add trailing zeros to a number:

# Integer value to be converted
i = 42 # Convert the integer to a string
s = str(i) # Use ljust to add trailing zeros, specifying the total width and the padding character ('0')
s_padded = s.ljust(5, '0') print(s_padded)
# Output: '42000'

In this example:

  • str(i) converts the integer i to a string.
  • s.ljust(5, '0') pads the string s with zeros to the right to make the total width 5 characters.

This is the most Pythonic way to accomplish this challenge.

Method 2: Format String


The second method uses the format string feature in Python 3+ called f-strings or replacement fields.

? Info: In Python, f-strings allow for the embedding of expressions within strings by prefixing a string with the letter "f" or "F" and enclosing expressions within curly braces {}. The expressions within the curly braces in the f-string are evaluated, and their values are inserted into the resulting string. This allows for a concise and readable way to include variable values or complex expressions within string literals.

The following f-string converts an integer i to a string while adding trailing zeros to a given integer:

# Integer value to be converted
i = 42 # Convert the integer to a string and then use format to add trailing zeros
s1 = f'{str(i):<5}'
s1 = s1.replace(" ", "0") # replace spaces with zeros print(s1)
# 42000

The code f'{str(i):<5}' first converts the integer i to a string. The :<5 format specifier aligns the string to the left and pads with spaces to make the total width 5. Then we replace the padded spaces with zeros using the string.replace() function.

Method 3: List Comprehension


Many Python coders don’t quite get the f-strings and the ljust() method shown in Methods 1 and 2. If you don’t have time to learn them, you can also use a more standard way based on string concatenation and list comprehension.

# Method 3: List Comprehension
s3 = str(42)
n = len(s3)
s3 = s3 + '0' * (5-len(s3))
print(s3)
# 42000

You first convert the integer to a basic string. Then, you concatenate the integer’s string representation to the string of 0s, filled up to n=5 characters. The asterisk operator creates a string of 5-len(s3) zeros here.

Programmer Humor


“Real programmers set the universal constants at the start such that the universe evolves to contain the disk with the data they want.”xkcd

? Recommended: Python Int to String with Leading Zeros

The post Python Int to String with Trailing Zeros appeared first on Be on the Right Side of Change.



https://www.sickgaming.net/blog/2023/09/...ing-zeros/

Print this item

  [Tut] PHP QR Code Generator with chillerlan-php-qrcode Library
Posted by: xSicKxBot - 12-01-2025, 05:47 PM - Forum: PHP Development - No Replies

[Tut] PHP QR Code Generator with chillerlan-php-qrcode Library

by Vincy. Last modified on June 15th, 2023.

This tutorial will create an example for generating a QR code using PHP. This example uses the Chillerlan QR code library. It is a PHP library with advanced features for creating QR codes, bar codes, and more.

There are two examples in this project. The first is a basic use-case scenario, and the second is an advanced example.

Both will help familiarize this library to send data for QR code rendering.

Quick Example


<?php
require_once '../vendor/autoload.php'; use chillerlan\QRCode\QRCode; // Core class for generating the QR code
$qrCode = new QRCode(); // data for which the QR code will be generated
$data = 'www.phppot.com'; // QR code image generation using render function
// it returns the an image resource.
$qrCodeImage = $qrCode->render($data); // Show the generated QR code image on screen
// following header is necessary to show image output
// in the browser
header('Content-Type: image/png');
imagepng($qrCodeImage);
imagedestroy($qrCodeImage);

The above code is a quick example of generating a Chillerlan QR code. You should use Composer to download the chillerlan dependency.

This example imports the library class and gives the data to generate the QR code.

The render() function passes the data to the library with which it will output the QR code image. This output can be returned to a browser or can be saved as a file.

In a previous article, we learned how to render the generated QR code to the browser.

chillerlan php qrcode

Download via composer


Run the following command in your terminal to install this Chillerlan PHP library.

composer require chillerlan/php-qrcode

qrcode project structure

Example 2 – How to configure size, EC level, scale


More configurations help to adjust the QR code quality without affecting readability.  The below parameters are used, which override the default configurations.

  • The version is to set the size of a QR code.
  • ECC level to set the possible values(L, M, Q, H). It is the damage tolerance percentage. We have seen it when coding with phpqrcode library.
  • Scale sets the size of a QR code pixel. The maximum size increases the QR code’s quality.

This library has the QROptions class to set the configurations explicitly. When initiating this class, the code below prepares an array of {version, eccLeverl …} options.

This QROptions instance generates the QRCode object to call the render() action handler. As in the above example, the render() uses the data and bundles it into the QR code binary.

<?php
require_once '../vendor/autoload.php'; use chillerlan\QRCode\QRCode;
use chillerlan\QRCode\QROptions; // data to embed in the QR code image
$data = 'www.phppot.com'; // configuration options for QR code generation
// eccLevel - Error correction level (L, M, Q, H)
// scale - QR code pixe size
// imageBase64 - output as image resrouce or not
$options = new QROptions([ 'version' => 5, 'eccLevel' => QRCode::ECC_H, 'scale' => 5, 'imageBase64' => true, 'imageTransparent' => false, 'foregroundColor' => '#000000', 'backgroundColor' => '#ffffff'
]); // Instantiating the code QR code class
$qrCode = new QRCode($options); // generating the QR code image happens here
$qrCodeImage = $qrCode->render($data); header('Content-Type: image/png');
imagepng($qrCodeImage);
imagedestroy($qrCodeImage);

Chillerlan PHP library


This is one of the popular QR Code generators in PHP. It has clean and easily understandable code with proper modularity.

Some of its features are listed below. This feature list represents the capability of being a component of a PHP application.

Features


  • Creates QR Codes with an improved Model, Version, ECC level, and more configuration
  • It supports encoding numeric, alphanumeric, 8-bit binary, and more.
  • It supports QR code output in GD, ImageMagick, SVG markup, and more formats.
  • It provides QR code readers using GD and ImageMagick libraries.

More about QR code


Hereafter we will see more about the QR code and its evolution,  advantages, and usage scenarios.

The QR code, or the quick response code, is a two-dimensional (2D) bar code. The linked article has the code to generate a barcode using PHP.

The QR code is a Japanese invention for the automotive industry. Later it spreads to more domains. Some of the commonly used places are,

  • Marketing
  • Linking to service providers
  • Information sharing
  • Online payments.

It provides easy access to online information through digital scanners. The QR code contains encoded data that can be decoded with digital scanning. It shares the information, links the service provider or prompts for the payment initiation after scanning.

Example usages of QR code generation and scanning


  • It shows payee details to ensure and allows one to enter the amount to make a mobile payment.
  • It facilitates storing location and contact details. It is for marking locations in the Google map while scanning.
  • When reading the QR code, the application will download v-cards using the contact details stored.
  • The app developing company shows QR codes in the app store to download mobile apps.

Download

↑ Back to Top



https://www.sickgaming.net/blog/2023/06/...e-library/

Print this item

  What’s new in Red Hat Dependency Analytics
Posted by: xSicKxBot - 12-01-2025, 01:23 AM - Forum: Java Language, JVM, and the JRE - No Replies

What’s new in Red Hat Dependency Analytics

We are excited to announce a new release of Red Hat Dependency Analytics, a solution that enables developers to create better applications by evaluating and adding high-quality open source components, directly from their IDE.

Red Hat Dependency Analytics helps your development team avoid security and licensing issues when building your applications. It plugs into the developer’s IDE, automatically analyzes your software composition, and provides recommendations to address security holes and licensing problems that your team may be missing.

Without further ado, let’s jump into the new capabilities offered in this release. This release includes a new version of the IDE plugin and the server-side analysis service hosted by Red Hat.

Support for Python applications


Along with Java (maven) and JavaScript (npm), Dependency Analytics now offers its full set of capabilities for Python (PyPI) applications. From your IDE, you can perform the vulnerability and license analysis of the “requirements.txt” file of your Python application, incorporate the recommended fixes, and generate the stack analysis report for more details.

Software composition analysis based on current vulnerability data


An estimated 15,000 open source packages get updated every day. On average, three new vulnerabilities get posted every day across JavaScript (npm) and Python (PyPi) packages. With this new release, the server-side analysis service hosted by Red Hat automatically processes the daily updates to open source packages that it is tracking. The hosted service also automatically ingests new vulnerability data posted to National Vulnerability Database (NVD) for JavaScript and Python packages. This allows the IDE plugin and API calls to provide source code analysis based on current vulnerability and release data.

Analyze transitive dependencies


In addition to the direct dependencies included in your application, Dependency Analytics now leverages the package managers to discover and add the dependencies of those dependencies, called “transitive” dependencies, to the dependency graph of your application. Analysis of your application is performed across the whole graph model and recommendations for fixes are provided across the entire set of dependencies.

Recommendations about complementary open source libraries


With this release, Dependency Analytics looks to recommend high-quality open source libraries that are complementary to the dependencies included in your application. The machine learning technology of the hosted service collects and analyzes various statistics on GitHub to curate a list of high-quality open source libraries that can be added to the current set of dependencies to augment your application. You can provide your feedback about the add-on libraries by clicking on the “thumbs-up” or “thumbs-down” icons shown for each recommendation. Your feedback is automatically processed to improve the quality of the recommendations.

IDE plugin support


The Dependency Analytics IDE plugin is now available for VS Code, Eclipse Che, and any JetBrains IDE, including IntelliJ and PyCharm.

We will continuously release new updates to our Dependency Analytics solution so you can minimize the delays in delivery of your applications due to last-minute security and licensing related issues.

Stay tuned for further updates; we look forward to your feedback about Dependency Analytics.

Share

The post What’s new in Red Hat Dependency Analytics appeared first on Red Hat Developer.



https://www.sickgaming.net/blog/2019/10/...analytics/

Print this item

  [Tut] How to Install xlrd in Python?
Posted by: xSicKxBot - 12-01-2025, 01:23 AM - Forum: Python - No Replies

[Tut] How to Install xlrd in Python?

5/5 – (2 votes)

The Python xlrd library reads data and formatting information from Excel files in the historical .xls format. Note that it won’t read anything other than .xls files.

pip install xlrd

The Python xlrd library is among the top 100 Python libraries, with more than 17,375,582 downloads. This article will show you everything you need to install this in your Python environment.

? Library Link
? Other Excel Python Libraries

Alternatively, you may use any of the following commands to install xlrd, depending on your concrete environment. One is likely to work!

? If you have only one version of Python installed:
pip install xlrd ? If you have Python 3 (and, possibly, other versions) installed:
pip3 install xlrd ? If you don't have PIP or it doesn't work
python -m pip install xlrd
python3 -m pip install xlrd
? If you have Linux and you need to fix permissions (any one):
sudo pip3 install xlrd
pip3 install xlrd
--user
? If you have Linux with apt
sudo apt install xlrd ? If you have Windows and you have set up the py alias
py -m pip install xlrd ? If you have Anaconda
conda install -c anaconda xlrd ? If you have Jupyter Notebook
!pip install xlrd
!pip3 install xlrd

Let’s dive into the installation guides for the different operating systems and environments!

How to Install xlrd on Windows?


  1. Type "cmd" in the search bar and hit Enter to open the command line.
  2. Type “pip install xlrd” (without quotes) in the command line and hit Enter again. This installs xlrd for your default Python installation.
  3. The previous command may not work if you have both Python versions 2 and 3 on your computer. In this case, try "pip3 install xlrd" or “python -m pip install xlrd“.
  4. Wait for the installation to terminate successfully. It is now installed on your Windows machine.

Here’s how to open the command line on a (German) Windows machine:

Open CMD in Windows

First, try the following command to install xlrd on your system:

pip install xlrd

Second, if this leads to an error message, try this command to install xlrd on your system:

pip3 install xlrd

Third, if both do not work, use the following long-form command:

python -m pip install xlrd

The difference between pip and pip3 is that pip3 is an updated version of pip for Python version 3. Depending on what’s first in the PATH variable, pip will refer to your Python 2 or Python 3 installation—and you cannot know which without checking the environment variables. To resolve this uncertainty, you can use pip3, which will always refer to your default Python 3 installation.

How to Install xlrd on Linux?


You can install xlrd on Linux in four steps:

  1. Open your Linux terminal or shell
  2. Type “pip install xlrd” (without quotes), hit Enter.
  3. If it doesn’t work, try "pip3 install xlrd" or “python -m pip install xlrd“.
  4. Wait for the installation to terminate successfully.

The package is now installed on your Linux operating system.

How to Install xlrd on macOS?


Similarly, you can install xlrd on macOS in four steps:

  1. Open your macOS terminal.
  2. Type “pip install xlrd” without quotes and hit Enter.
  3. If it doesn’t work, try "pip3 install xlrd" or “python -m pip install xlrd“.
  4. Wait for the installation to terminate successfully.

The package is now installed on your macOS.

How to Install xlrd in PyCharm?


Given a PyCharm project. How to install the xlrd library in your project within a virtual environment or globally? Here’s a solution that always works:

  • Open File > Settings > Project from the PyCharm menu.
  • Select your current project.
  • Click the Python Interpreter tab within your project tab.
  • Click the small + symbol to add a new library to the project.
  • Now type in the library to be installed, in your example "xlrd" without quotes, and click Install Package.
  • Wait for the installation to terminate and close all pop-ups.

Here’s the general package installation process as a short animated video—it works analogously for xlrd if you type in “xlrd” in the search field instead:


Make sure to select only “xlrd” because there may be other packages that are not required but also contain the same term (false positives):

How to Install xlrd in a Jupyter Notebook?


To install any package in a Jupyter notebook, you can prefix the !pip install my_package statement with the exclamation mark "!". This works for the xlrd library too:

!pip install my_package

This automatically installs the xlrd library when the cell is first executed.

How to Resolve ModuleNotFoundError: No module named ‘xlrd’?


Say you try to import the xlrd package into your Python script without installing it first:

import xlrd
# ... ModuleNotFoundError: No module named 'xlrd'

Because you haven’t installed the package, Python raises a ModuleNotFoundError: No module named 'xlrd'.

To fix the error, install the xlrd library using “pip install xlrd” or “pip3 install xlrd” in your operating system’s shell or terminal first.

See above for the different ways to install xlrd in your environment. Also check out my detailed article:

? Recommended: [Fixed] ModuleNotFoundError: No module named ‘xlrd’

Improve Your Python Skills


If you want to keep improving your Python skills and learn about new and exciting technologies such as Blockchain development, machine learning, and data science, check out the Finxter free email academy with cheat sheets, regular tutorials, and programming puzzles.

Join us, it’s fun! ?

✅ Recommended: Python Excel – Basic Worksheet Operations

The post How to Install xlrd in Python? appeared first on Be on the Right Side of Change.



https://www.sickgaming.net/blog/2023/09/...in-python/

Print this item

  [Tut] Web Scraping with PHP – Tutorial to Scrape Web Pages
Posted by: xSicKxBot - 12-01-2025, 01:21 AM - Forum: PHP Development - No Replies

[Tut] Web Scraping with PHP – Tutorial to Scrape Web Pages

by Vincy. Last modified on July 21st, 2023.

Web scraping is a mechanism to crawl web pages using software tools or utilities. It reads the content of the website pages over a network stream.

This technology is also known as web crawling or data extraction. In a previous tutorial, we learned how to extract pages by its URL.
View Demo

There are more PHP libraries to support this feature. In this tutorial, we will see one of the popular web-scraping components named DomCrawler.

This component is underneath the PHP Symfony framework. This article has the code for integrating and using this component to crawl web pages.

web scraping php

We can also create custom utilities to scrape the content from the remote pages. PHP allows built-in cURL functions to process the network request-response cycle.

About DomCrawler


The DOMCrawler component of the Symfony library is for parsing the HTML and XML content.

It constructs the crawl handle to reach any node of an HTML tree structure. It accepts queries to filter specific nodes from the input HTML or XML.

It provides many crawling utilities and features.

  1. Node filtering by XPath queries.
  2. Node traversing by specifying the HTML selector by its position.
  3. Node name and value reading.
  4. HTML or XML insertion into the specified container tag.

Steps to create a web scraping tool in PHP


  1. Install and instantiate an HTTP client library.
  2. Install and instantiate the crawler library to parse the response.
  3. Prepare parameters and bundle them with the request to scrape the remote content.
  4. Crawl response data and read the content.

In this example, we used the HTTPClient library for sending the request.

Web scraping PHP example


This example creates a client instance and sends requests to the target URL. Then, it receives the web content in a response object.

The PHP DOMCrawler parses the response data to filter out specific web content.

In this example, the crawler reads the site title by parsing the h1 text. Also, it parses the content from the site HTML filtered by the paragraph tag.

The below image shows the example project structure with the PHP script to scrape the web content.

web scraping php project structure

How to install the Symfony framework library


We are using the popular Symfony to scrape the web content. It can be installed via Composer.
Following are the commands to install the dependencies.

composer require symfony/http-client symfony/dom-crawler
composer require symfony/css-selector

After running these composer commands, a vendor folder can map the required dependencies with an autoload.php file. The below script imports the dependencies by this file.

index.php

<?php require 'vendor/autoload.php'; use Symfony\Component\HttpClient\HttpClient;
use Symfony\Component\DomCrawler\Crawler; $httpClient = HttpClient::create(); // Website to be scraped
$website = 'https://example.com'; // HTTP GET request and store the response
$httpResponse = $httpClient->request('GET', $website);
$websiteContent = $httpResponse->getContent(); $domCrawler = new Crawler($websiteContent); // Filter the H1 tag text
$h1Text = $domCrawler->filter('h1')->text();
$paragraphText = $domCrawler->filter('p')->each(function (Crawler $node) { return $node->text();
}); // Scraped result
echo "H1: " . $h1Text . "\n";
echo "Paragraphs:\n";
foreach ($paragraphText as $paragraph) { echo $paragraph . "\n";
}
?>

Ways to process the web scrapped data


What will people do with the web-scraped data? The example code created for this article prints the content to the browser. In an actual application, this data can be used for many purposes.

  1. It gives data to find popular trends with the scraped news site contents.
  2. It generates leads for showing charts or statistics.
  3. It helps to extract images and store them in the application’s backend.

If you want to see how to extract images from the pages, the linked article has a simple code.

Caution


Web scraping is theft if you scrape against a website’s usage policy.  You should read a website’s policy before scraping it. If the terms are unclear, you may get explicit permission from the website’s owner. Also, commercializing web-scraped content is a crime in most cases. Get permission before doing any such activities.

Before crawling a site’s content, it is essential to read the website terms. It is to ensure that the public can be subject to scraping.

People provide API access or feed to read the content. It is fair to do data extraction with proper API access provision. We have seen how to extract the title, description and video thumbnail using YouTube API.

For learning purposes, you may host a dummy website with lorem ipsum content and scrape it.
View Demo

↑ Back to Top



https://www.sickgaming.net/blog/2023/06/...web-pages/

Print this item

  (Free Game Key) Epic Games | Universe for Sale
Posted by: xSicKxBot - 11-28-2025, 07:13 AM - Forum: Deals or Specials - No Replies

(Free Game Key) Epic Games | Universe for Sale

  1. Go to the store page
  2. Click the GET button
  3. Verify the game is free
Universe for Sale

Free till December 4, 16:00 UTC.


https://steamcommunity.com/groups/GrabFr...1455151462

Print this item

 
Latest Threads
[Tut] Google’s SynthID is...
Last Post: xSicKxBot
1 hour ago
(Free Game Key) Steam - W...
Last Post: xSicKxBot
2 hours ago
[Tut] How to Use ChatGPT ...
Last Post: xSicKxBot
Today, 02:30 AM
[Tut] React Charts and Gr...
Last Post: xSicKxBot
Today, 02:30 AM
(Indie Deal) New Giveaway...
Last Post: xSicKxBot
Today, 01:27 AM
(Free Game Key) Steam - W...
Last Post: xSicKxBot
Today, 01:27 AM
[Tut] This Finviz Screene...
Last Post: xSicKxBot
Yesterday, 09:54 AM
[Tut] Integrate Google Ma...
Last Post: xSicKxBot
Yesterday, 09:54 AM
Forza Horizon 5 Game Save...
Last Post: bozsmuymuy
Yesterday, 03:36 AM
[Tut] Restoring Images wi...
Last Post: xSicKxBot
12-10-2025, 05:15 PM

Forum software by © MyBB Theme © iAndrew 2016