Seaching for files or even folders manually is a tedious task. I could use an in-built search function, but even that doesnot search every nook and cranny. Have you ever been in a situation where you know the name of the file or the directory but you don't remember where you saved it. You start searching for it manually since it didnot show up in the in-built search feature. Well, I have been there too. It's like finding a needle in a haystack. You need to find a file/folder in gazillions of files you have and their structure is just ewwww. This dull task made me create a pythoh script that goes and searches for the file/folder you want starting from the path you desire.

parser = argparse.ArgumentParser()
parser.add_argument("--file", type=str, help="File to search")
parser.add_argument("--folder", type=str, help="Folder to search")
parser.add_argument("--path", type=str, help="Path from where the search will start")
args = parser.parse_args()

search_file = args.file
search_folder = args.folder
search_path = '.'
if args.path and os.path.isdir(args.path):
    search_path = args.path

for path, dirs, files in os.walk(search_path):
    if search_file in files:
        print(f'{os.path.join(path, search_file)}')
    elif search_folder in dirs:
        print(f'{os.path.join(path, search_folder)}')

python3 find.py --file secretfile --path /home

Effortlessly Find Files and Folders with Python

Map and Reduce tasks are fundamental concepts in distributed and parallel computing, often used in frameworks like MapReduce or Apache Spark. They are designed to process large datasets efficiently across multiple nodes or processors.



The Map task applies a specified function (the "mapper") to each element in a dataset independently and in parallel.

It takes an input dataset and produces a set of intermediate key-value pairs.

Each element of the input dataset is processed independently, allowing for parallelization.

The output of the Map task is a collection of intermediate key-value pairs, where the keys are typically used for grouping related data together.

The Reduce task applies a specified function (the "reducer") to a group of intermediate key-value pairs with the same key, producing a smaller set of key-value pairs.

It takes the output of the Map task (the intermediate key-value pairs) as input.

The reducer function is applied to each group of key-value pairs with the same key.

The output of the Reduce task is typically a reduced or aggregated version of the data, often representing some summary or analysis of the original dataset.


Map and Reduce tasks are fundamental concepts in distributed and parallel computing, often used in frameworks like MapReduce or Apache Spark. They are designed to process large datasets efficiently across multiple nodes or processors.

Here's a simplified flow of how Map and Reduce tasks work together:

The input dataset is divided into smaller chunks.

Each chunk is processed by a separate Map task, applying the mapper function to each element.

The mapper function produces intermediate key-value pairs.

The intermediate key-value pairs from all Map tasks are shuffled and sorted based on their keys. This ensures that all key-value pairs with the same key are grouped together.

The shuffled and sorted intermediate key-value pairs are input to the Reduce tasks.

Each Reduce task processes a group of key-value pairs with the same key, applying the reducer function to produce the final output.

Map and Reduce tasks provide a powerful framework for processing large-scale data in a distributed and parallel manner, enabling efficient computation on clusters of machines or multicore processors. They are widely used in various big data processing frameworks and have become essential tools in modern data analytics and processing pipelines.

from multiprocessing import Pool
from functools import reduce
import re

def map_function(line):
# Split the line into words and create a tuple (word, 1) for each word
words = re.findall(r'\b\w+\b', line.lower())
return [(word, 1) for word in words]

def reduce_function(item1, item2):
# Reduce function to sum up the counts for each word
word, count1 = item1
_, count2 = item2
return (word, count1 + count2)

if __name__ == "__main__":
# Read text from a file
with open("sample_text.txt", "r") as file:
lines = file.readlines()

# Map phase: count occurrences of each word
with Pool() as pool:
mapped_items = pool.map(map_function, lines)

# Flatten the list of mapped items
flattened_items = [item for sublist in mapped_items for item in sublist]

# Reduce phase: sum up the counts for each word
word_counts = reduce(reduce_function, flattened_items)

# Print the word counts
for word, count in word_counts:
print(f"{word}: {count}")


Exploring Map and Reduce: A Word Count Program Demonstration in Parallel Computing

The FP-Growth algorithm is a data mining technique for discovering frequent patterns in large datasets, particularly useful for association rule mining. It uses a compact structure called an FP-Tree to represent frequent items efficiently. The algorithm constructs this tree, mines it for frequent patterns recursively, and generates association rules. FP-Growth is known for its efficiency in handling large datasets compared to traditional methods like Apriori.

#Finding the frequent patterns with min support threshold=0.5

print("Generating rules with min confidence threshold=0.5")

FrequentPatterns=pyfpgrowth.find_frequent_patterns(transactions=transactions,support_threshold=0.5)

# Generating rules with min confidence threshold=0.8

print("Generating rules with min confidence threshold=0.8")

Rules=pyfpgrowth.generate_association_rules(patterns=FrequentPatterns,confidence_threshold=0.8)

NICT Club Blogs.

FP Growth Algorithm

More Stories

Effortlessly Find Files and Folders with Python

Exploring Map and Reduce: A Word Count Program Demonstration in Parallel Computing